In the previous post, we introduced task planning in robotics. This field broadly involves a set of planning techniques that are domain-independent: That is, we can design a domain which describes the world at some (typically high) level of abstraction, using some modeling language like PDDL. However, the planning algorithms themselves can be applied to any domain that can be modeled in that language, and critically, to solve any valid problem specification within that domain.

The key takeaway of the last post was that task planning is ultimately search. These search problems are often challenging and grow exponentially with the size of the problem, so it is no surprise that task planning is often symbolic: There are relatively few possible actions to choose from, with a relatively small set of finite parameters. Otherwise, search is prohibitively expensive even in the face of clever algorithms and heuristics.

Bridging the gap between this abstract planning space and the real world, which we deeply care about in robotics, is hard. In our example of a mobile robot navigating a household environment, this might look as follows: Given that we know two rooms are connected, a plan that takes our robot from room A to room B is guaranteed to execute successfully. Of course, this is not necessarily true. We might come up with a perfectly valid abstract plan, but then fail to generate a dynamically feasible motion plan through a narrow hallway, or fail to execute a perfectly valid motion plan on our real robot hardware.

This is where Task and Motion Planning (TAMP) comes in. What if our planner spends more effort deliberating about more concrete aspects of a plan before executing it? This presents a key tradeoff in more up-front computation, but a lower risk of failing at runtime. In this post we will explore a few things that differentiate TAMP from “plain” task planning, and dive into some detailed examples with the pyrobosim and PDDLStream software tools.

Some motivating examples

Before we formalize TAMP further, let’s consider some tricky examples you might encounter with purely symbolic planning in robotics applications.

In this first example, our goal is to pick up an apple. In purely symbolic planning where all actions have the same cost, there is no difference in navigating to table0 and table1, both of which have apples. However, you will notice that table0 is in an unreachable location. Furthermore, if we decide to embed navigation actions with a heuristic cost such as straight-line distance from the starting point, our heuristic below will prefer table0 because it’s closer to the robot’s starting position than table1.

It wouldn’t be until we try to refine our plan — for example, using a motion planner to search for a valid path to table0 in the unreachable room — that we would fail, have to update our planning domain to somehow flag that main_room and unreachable are disconnected, and then replan with this new information.

Pathological task planning example for goal-directed navigation.
Both table0 and table1 can lead to solving the goal specification of holding an apple, but table0 is completely unreachable.

In this second example, we want to place a banana on a desk. As with the previous example, we could choose to place this object on either desk0 or desk1. However, in the absence of more information — and especially if we continue to treat nearby locations as lower cost — we may plan to place banana0 on desk0 and fail to execute at runtime because of the other obstacles.

Here, some alternative solutions would include placing banana0 on desk1, or moving one of the other objects (water0 or water1) out of the way to enable banana0 to fit on desk0. Either way, we need some notion of collision checking to enable our planner to eliminate the seemingly optimal, but in practice infeasible, plan of simply placing the object on desk0.

Pathological task planning example for object manipulation.
Placing banana0 on either desk0 or desk1 will satisfy the goal, but desk0 has other objects that would lead to collisions. So, banana0 must either placed on desk1, or the objects need to be rearranged and/or moved elsewhere to allow banana0 to fit on desk0.

In both cases, what we’re missing from our purely symbolic task planner is the ability to consider the feasibility of abstract actions before spitting out a plan and hoping for the best. Specifically for embodied agents such as robots, which move in the physical world, symbolic plans need to be made concrete through motion planning. As seen in these two examples, what if our task planner also required the existence of a specific path to move between two locations, or a specific pose for placing objects in a cluttered space?

What is Task and Motion Planning?

In our examples, the core issue is that if our task planning domain is too abstract, a seemingly valid plan is likely to fail down the line when we call a completely decoupled motion planner to try execute some portion of that plan. So, task and motion planning is exactly as its name states — jointly thinking about tasks and motion in a single planner. As Garrett et al. put it in their 2020 survey paper, “TAMP actually lies between discrete “high-level” task planning and continuous “low-level” motion planning”.

However, there is no free lunch. When considering all the fine details up front in deliberative planning, search becomes expensive very quickly. In symbolic planning, an action may have a discrete, finite list of possible goals (let’s say somewhere around 5-10), so it may be reasonable to exhaustively search over these and find the one parameter that is optimal according to our model. When we start thinking about detailed motion plans that have a continuous parameter space spanning infinite possible solutions, this becomes intractable. So, several approaches to TAMP will apply sampling-based techniques to make planning work in continuous action spaces.

Another way to ensure TAMP is practical is to leverage hierarchy. One popular technique for breaking down symbolic planning into manageable pieces is Hierarchical Task Networks (HTNs). In these 2012 lecture slides, Nilufer Onder mentions “It would be a waste of time to construct plans from individual operators. Using the built-in hierarchy helps escape from exponential explosion.” An example of hierarchical planning is shown in the diagram below. Using this diagram, you can explore the benefits of hierarchy; for example, this planner would never have to even consider how to open a door if the abstract plan didn’t require going down the hallway.

An example of hierarchical planning for a robot, where high-level, or abstract, plans for a robot could be refined into lower-level, or concrete, actions.
Source: Automated Planning and Acting (2016)

Hierarchical planning is great in that it helps prune infeasible plans before spending time producing detailed, low-level plans. However, in this space the mythical downward refinement property is often cited. To directly quote the 1991 paper by Bacchus and Yang, this property states that “given that a concrete-level solution exists, every abstract solution can be refined to a concrete-level solution without backtracking across abstract levels”. This is not always (and I would argue rarely) achievable in robotics, so backtracking in hierarchical planning is largely unavoidable.

To this end, another strategy behind TAMP has to do with commitment in sampling parameters during search. In the literature, you will see many equivalent words thrown around, but I find the main distinction is between the following strategies:

• Early-commitment (or binding) strategies will sample action parameters from continuous space before search, effectively converting the problem to a purely discrete task planning problem.
• Least-commitment (or optimistic) strategies will instead come up with a purely symbolic plan skeleton. If that skeleton is feasible, then the necessary parameter placeholders are filled by sampling.

Flowcharts representing two extreme varieties of sampling-based TAMP.
*H-CSP = hybrid constraint satisfaction problem
Source: Garrett et al. (2020), Integrated Task and Motion Planning

Both strategies have advantages and disadvantages, and in practice modern TAMP methods will combine them in some way that works for the types of planning domains and problems being considered. Also, note that in the diagram above both strategies have a loop back to the beginning when a solution is not found; so backtracking remains an unavoidable part of planning.

One key paper that demonstrated the balance of symbolic search and sampling was Sampling-based Motion and Symbolic Action Planner (SMAP) by Plaku and Hager in 2010. Around the same time, in 2011, Leslie Kaelbling and Tomás Lozano-Pérez presented Hierarchical Planning in the Now (HPN), which combined hierarchy and sampling-based techniques for TAMP. However, the authors themselves admitted the sampling part left something to be desired. There is a great quote in this paper which foreshadows some of the other work that would come out of their lab:

“Because our domains are infinite, we cannot consider all instantiations of the operations. Our current implementation of suggesters only considers a small number of possible instantiations of the operations. We could recover the relatively weak properties of probabilistic completeness by having the suggesters be generators of an infinite stream of samples, and managing the search as a non-deterministic program over those streams.”

– Leslie pack kaelbling and Tomás Lozano-Pérez (2011), Hierarchical planning in the now.

Directly following this quote is the work their student Caelan Garrett took on — first in the creation of STRIPStream in 2017 and then PDDLStream in 2018. The astute reader will have noticed that PDDLStream is the actual software used in these blog posts, so take this “literature review” with this bias in mind, and keep reading if you want to learn more about TAMP with this specific tool.

If you want to know more about TAMP in general, I will refer you to two recent survey papers that I found useful:

• Integrated Task and Motion Planning, by Garrett et al. (2020)
• Combining task and motion planning: Challenges and Guidelines, by Mansouri et al. (2021)

Mobile robot example, revisited

To illustrate the benefits of integrated TAMP, we’ll continue the same mobile robotics example from the previous post. In this problem,

• The robot’s goal is to place the apple on the table.
• Navigation now requires coming up with a goal pose (which is a continuous parameter), as well the actual path from start to goal. For this example, we are using a Rapidly-exploring Random Tree (RRT), but you could swap for any other path-finding algorithm.
• Placing an object now requires sampling a valid pose that is inside the placement surface polygon and does not collide with other objects on that surface.

As you read the following list explaining this problem, make sure you scroll through the slideshow below to get a visual representation.

STEP 1: Looking at the state of the world, you can see how a purely symbolic task planner would output a relatively simple plan: pick the apple, move to the table, and place the apple on the table. In the context of TAMP, this now represents a plan skeleton which several parameters that are yet to be filled — specifically,

• ?pt is the pose of the robot when navigating to the table
• ?path is the actual output of our motion planner to get to ?pt
• ?pa-1 is the new pose of the apple when placed on the table (which follows from its initial pose ?pa-0)

STEP 2: To make the problem a little simpler, we made it such that every location has a discrete, finite set of possible navigation locations corresponding to the edges of its polygon. So looking at the table location, you see there are 4 possible navigation poses pt-T, pt-B, pt-L, and pt-R corresponding to the top, bottom, left, and right sides, respectively. Since this set of locations is relatively small, we can sample these parameters up front (or eagerly) at the start of search.

STEP 3: Our move action can now have different instantiations for the goal pose ?pt that are enumerated during search. This is in contrast with the ?path argument, which must be sampled by calling our RRT planner. We don’t want to do this eagerly because the space of paths is continuous, so we prefer to defer sampling of this parameter. If our action has a cost associated with the length of a path, we could imagine that the lowest-cost action would be to navigate to the left side of the table (pt-L), and some randomly sampled path (path42) may describe how we get there.

STEP 4: Next comes the place action, which now must include a valid collision-free pose for the apple on the table. Because of how we set up our problem, our robot cannot find a valid placement pose when approaching from the left side of the table. So, we must backtrack.

STEP 5: After backtracking, we need to find an alternative navigation pose for the table (?pt). Given our environment, the only other feasible location is the bottom side of the table (pt-b), as the walls block the robot from the top and right sides and it would be impossible to find a valid path with our RRT. However, when the robot is at the bottom side of the table, it can also sample a valid placement pose! In our example, the placeholder ?pa-1 is therefore satisfied with some randomly sampled pose pa29.

STEP 6: … And there you have it! A valid plan that defines a sequence of symbolic actions (pick, move, place) along with the necessary navigation pose, path to that pose, and placement location for the apple. It’s not optimal, but it is probabilistically complete!

(1/6) By being optimistic about all the continuous parameters related to motion, we can reach a potential goal state with relative ease.

(2/6) Since the navigation poses around the desk and the table are finite, we can sample them eagerly; that is, we enumerate all options up front in planning.

(3/6) Once we commit to a navigation pose around the table, we can continue filling in our plan by sampling a feasible trajectory from the robot’s current pose to the target pose at the table.

(4/6) Next, we need to sample a placement pose for the apple. Suppose in this case we fail to sample a collision-free solution based on the robot’s current location.

(5/6) This means we need to backtrack and consider a different navigation pose, thereby a different motion plan to this new pose.

(6/6) From this new pose, even though the trajectory is longer and therefore higher-cost, we can sample a valid placement pose for the apple and finally complete our task and motion plan.

Now, suppose we change our environment such that we can only approach the table from the left side, so there is no way to directly find a valid placement pose for the apple. Using the same planner, we should eventually converge on a task and motion plan that rearranges the objects world — that is, it requires moving one of the other objects on the table to make room for the apple.

Implementing TAMP with PDDLStream

We will now revisit our pathological examples from the beginning of this post. To do this, we will use PDDLStream for planning and pyrobosim as a simple simulation platform. For quick background on PDDLStream, you may refer to this video.

The key idea behind PDDLStream is that it extends PDDL with a notion of streams (remember the earlier quote from the Hierarchical Planning in the Now paper?). Streams are generic, user-defined Python functions that sample continuous parameters such as a valid sample certifies that stream and provides any necessary predicates that (usually) act as preconditions for actions. Also, PDDLStream has an adaptive technique that balances exploration (searching for discrete task plans) vs. exploitation (sampling to fill in continuous parameters).

Goal-directed navigation

We can use PDDLStream to augment our move action such that it includes metric details about the world. As we saw in our illustrative example, we now must factor in the start and goal pose of our robot, as well as a concrete path between those poses.

As additional preconditions for this action, we must ensure that:

• The navigation pose is valid given the target location (NavPose)
• There must be a valid path from the start to goal pose (Motion)

Additionally, we are able to now use more realistic costs for our action by calculating the actual length of our path produced by the RRT! The separate file describing the streams for this action may look as follows. Here, the s-navpose stream certifies the NavPose predicate and the s-motion stream certifies the Motion predicate.

The Python implementations for these functions would then look something like this. Notice that the get_nav_poses function returns a finite set of poses, so the output is a simple Python list. On the other hand, sample_motion can continuously spit out paths from our RRT, and it implemented as a generator:

Putting this new domain and streams together, we can solve our first pathological example from the introduction. In the plan below, the robot will compute a path to the farther away, but reachable room to pick up an apple and satisfy the goal.

Object manipulation

Similarly, we can extend our place action to now include the actual poses of objects in the world. Specifically,

• The ?placepose argument defines the target pose of the object.
• The Placeable predicate is certified by a s-place stream.
• The IsCollisionFree predicate is actually a derived predicate that checks individual collisions between the target object and all other objects at that location.
• Each individual collision check is determined by the CollisionFree predicate, which is certified by a t-collision-free steam.

The Python implementation for sampling placement poses and checking for collisions may look as follows. Here, sample_place_pose is our generator for placement poses, whereas test_collision_free is a simple Boolean (true/false) check.

By expanding our domain to reason about the feasibility of object placement, we can similarly solve the second pathological example from the introduction. In the first video, we have an alternative location desk1 where the robot can place the banana and satisfy the goal.

In the second video, we remove the alternative desk1. The same task and motion planner then produces a solution that involves picking up one of the objects on desk0 to make room to later place the banana there.

You can imagine extending this to a more realistic system — that is, one that is not a point robot and has an actual manipulator — that similarly checks the feasibility of a motion plan for picking and placing objects. While it wasn’t the main focus of the work, our Active Learning of Abstract Plan Feasibility work did exactly this with PDDLStream. Specifically, we used RRTs to sample configuration-space paths for a Franka Emika Panda arm and doing collision-checking using a surrogate model in PyBullet!

Conclusion

In this post we introduced the general concept of task and motion planning (TAMP). In theory, it’s great to deliberate more — that is, really think more about the feasibility of plans down to the metric level — but with that comes more planning complexity. However, this can pay off in that it reduces the risk of failing in the middle of executing a plan and having to stop and replan.

We introduced 3 general principles that can make TAMP work in practice:

• Hierarchy, to determine the feasibility of abstract plans before planning at a lower level of refinement.
• Continuous parameter spaces, and techniques like sampling to make this tractable.
• Least-commitment strategies, to come up with symbolic plan skeletons before spending time with expensive sampling of parameters.

We then dug into PDDLStream as one tool for TAMP, which doesn’t do much in the way of hierarchy, but certainly tackles continuous parameter spaces and least-commitment strategies for parameter binding. We went through a few examples using pyrobosim, but you can access the full set of examples in the pyrobosim documentation for TAMP.

The PDDLStream repository has many more examples that you can check out. And, of course, there are many other task and motion planners out there that focus on different things — such as hierarchy without continuous parameters, or factoring in other common objectives such as temporal aspects and resource consumption.

Hope you have enjoyed these posts! If the tools shown here give you any cool ideas, I would love to hear about them, so feel free to reach out.

You can read the original article at Roboticseabass.com.

Suppose we have a robot in a simple world like the one below. Let’s consider commanding our robot to perform a task such as “take the apple from the shelf and put it on the table”.

Simple task planning example world: A robot can move between a finite set of locations, and can pick and place objects at those locations.

I would argue we humans have pretty good intuition for how a robot could achieve this task. We could describe what the robot should do by breaking the solution down into individual actions. For example:

• Move to the shelf
• Pick up the apple
• Move back to the table
• Place the apple

This is fine for our simple example — and in fact, is also fine for many real-world robotics applications — but what if the task gets more complicated? Suppose we add more objects and locations to our simple world, and start describing increasingly complex goals like “make sure all the apples are on the table and everything else is in the garbage bin”? Furthermore, what if we want to achieve this in some optimal manner like minimizing time, or number of total actions? Our reasoning quickly hits its limit as the problem grows, and perhaps we want to consider letting a computer do the work.

This is what we often refer to as task planning: Autonomously reasoning about the state of the world using an internal model and coming up a sequence of actions, or a plan, to achieve a goal.

In my opinion, task planning is not as popular as other areas in robotics you might encounter because… quite frankly, we’re just not that good at robotics yet! What I mean is that robots are complicated, and many commercially available systems are automating simple, repetitive tasks that don’t require this level of high-level planning — it would be overkill! There are many lower-level problems to solve in robotics such as standing up good hardware, robust sensing and actuation, motion planning, and keeping costs down. While task planning skews towards more academic settings for this reason, I eagerly await the not-so-distant future where robots are even more capable and the entire industry is thinking more about planning over longer horizons and increasingly sophisticated tasks.

In this post, I will introduce the basic pieces behind task planning for robotics. I will focus on Planning Domain Definition Language (PDDL) and take you through some basic examples both conceptually and using the pyrobosim tool.

Defining a planning domain

Task planning requires a model of the world and how an autonomous agent can interact with the world. This is usually described using state and actions. Given a particular state of the world, our robot can take some action to transition to another state.

Over the years, there have been several languages used to define domains for task planning. The first task planning language was the STanford Research Institute Problem Solver (STRIPS) in 1971, made popular by the Shakey project.

Since then, several related languages for planning have come up, but one of the most popular today is Planning Domain Definition Language (PDDL). The first PDDL paper was published in 1998, though there have been several enhancements and variations tacked on over the years. To briefly describe PDDL, it’s hard to beat the original paper.

“PDDL is intended to express the “physics” of a domain, that is, what predicates there are, what actions are possible, what the structure of compound actions is, and what the effects of actions are.”

– GHALLAB ET AL. (1998), PDDL – THE PLANNING DOMAIN DEFINITION LANGUAGE

The point of languages like PDDL is that they can describe an entire space of possible problems where a robot can take the same actions, but in different environments and with different goals. As such, the fundamental pieces of task planning with PDDL are a task-agnostic domain and a task-specific problem.

Using our robot example, we can define:

• Domain: The task-agnostic part
  • Predicates: (Robot ?r), (Object ?o), (Location ?loc), (At ?r ?loc), (Holding ?r ?o), etc.
  • Actions: move(?r ?loc1 ?loc2), pick(?r ?o ?loc), place(?r ?o ?loc)
• Problem: The task-specific part
  • Objects: (Robot robot), (Location shelf), (Location table), (Object apple)
  • Initial state: (HandEmpty robot), (At robot table), (At apple shelf)
  • Goal specification: (At apple table)

Since domains describe how the world works, predicates and actions have associated parameters, which are denoted by ?, whereas a specific object does not have any special characters to describe it. Some examples to make this concrete:

• (Location ?loc) is a unary predicate, meaning it has one parameter. In our example, shelf and table are specific location instances, so we say that (Location table) and (Location shelf) are part of our initial state and do not change over time.
• (At ?r ?loc) is a binary predicate, meaning it has two parameters. In our example, the robot may begin at the table, so we say that (At robot table) is part of our initial state, though it may be negated as the robot moves to another location.

PDDL is an action language, meaning that the bulk of a domain is defined as actions (sometimes referred to as operators) that interact with our predicates above. Specifically, an action contains:

• Parameters: Allow the same type of action to be executed for different combinations of objects that may exist in our world.
• Preconditions: Predicates which must be true at a given state to allow taking the action from that state.
• Effects: Changes in state that occur as a result of executing the action.

For our robot, we could define a move action as follows:

(:action move
:parameters (?r ?loc1 ?loc2)
:precondition (and (Robot ?r)
(Location ?loc1)
(Location ?loc2)
(At ?r ?loc1))
:effect (and (At ?r ?loc2)
(not (At ?r ?loc1)))
)

In our description of move, we have

• Three parameters ?r ?loc1, and ?loc2.
• Unary predicates in the preconditions that narrow down the domain of parameters that make an action feasible — ?r must be a robot and ?loc1 and ?loc2 must be locations, otherwise the action does not make sense.
• Another precondition that is state-dependent: (At ?r ?loc1). This means that to perform a move action starting at some location ?loc1, the robot must already be in that location.

Note that in some cases, PDDL descriptions allow for typing, which lets you define the domain of actions inline with the parameters rather than explicitly including them as unary predicates — for example, :parameters ?r – Robot ?loc1 – Location ?loc2 – Location … but this is just syntactic sugar.

Similarly, the effects of the action can add new predicates or negate existing ones (in STRIPS these were specified as separate addition and deletion lists). In our example, after performing a move action, the robot is no longer at its previous location ?loc1 and instead is at its intended new location ?loc2.

A similar concept can be applied to other actions, for example pick and place. If you take some time to process the PDDL snippets below, you will hopefully get the gist that our robot can manipulate an object only if it is at the same location as that object, and it is currently not holding something else.

(:action pick
:parameters (?r ?o ?loc)
:precondition (and (Robot ?r)
(Obj ?o)
(Location ?loc)
(HandEmpty ?r)
(At ?r ?loc)
(At ?o ?loc))
:effect (and (Holding ?r ?o)
(not (HandEmpty ?r))
(not (At ?o ?loc)))
)

(:action place
:parameters (?r ?o ?loc)
:precondition (and (Robot ?r)
(Obj ?o)
(Location ?loc)
(At ?r ?loc)
(not (HandEmpty ?r))
(Holding ?r ?o))
:effect (and (HandEmpty ?r)
(At ?o ?loc)
(not (Holding ?r ?o)))
)

So given a PDDL domain, we can now come up a plan, or sequence of actions, to solve various types of problems within that domain … but how is this done in practice?

Task planning is search

There is good reason for all this overhead in defining a planning domain and a good language to express it in: At the end of the day, task planning is solved using search algorithms, and much of the literature is about solving complex problems as efficiently as possible. As task planning problems scale up, computational costs increase at an alarming rate — you will often see PSPACE-Complete and NP-Complete thrown around in the literature, which should make planning people run for the hills.

State-space search

One way to implement task planning is using our model to perform state-space search. Given our problem statement, this could either be:

• Forward search: Start with the initial state and expand a graph until a goal state is reached.
• Backward search: Start with the goal state(s) and work backwards until the initial state is reached.

Using our example, let’s see how forward state-space search would work given the goal specification (At apple table):

(1/6) In our simple world, the robot starts at the table. The only valid action from here is to move to the shelf.

(2/6) After moving to the shelf, the robot could either pick the apple or move back to the table. Since moving back to the table leads us to a state we have already seen, we can ignore it.

(3/6) After picking the apple, we could move back to the table or place the apple back on the shelf. Again, placing the apple would lead us to an already visited state.

(4/6) Once the robot is at the table while holding the apple, we can place the apple on the table or move back to the shelf.

(5/6) At this point, if we place the apple on the table we reach a goal state! The sequence of actions leading to the state is our resulting plan.

(6/6) As the number of variables increases, the search problem (and time to find a solution) grows exponentially.

Deciding which states to expand during search could be purely based on a predetermined traversal strategy, using standard approaches like breadth-first search (BFS), depth-first search (DFS), and the like. Whether we decide to add costs to actions or treat them all as unit cost (that is, an optimal plan simply minimizes the total number of actions), we could instead decide to use greedy or hill-climbing algorithms to expand the next state based on minimum cost. And finally, regardless of which algorithm we use, we probably want to keep track of states we have already previously visited and prune our search graph to prevent infinite cycles and expanding unnecessary actions.

In motion planning, we often use heuristics during search; one common example is the use of A* with the straight-line distance to a goal as an admissible heuristic. But what is a good heuristic in the context of task planning? How would you define the distance to a goal state without a handy metric like distance? Indeed, a great portion of the literature focuses on just this. Methods like Heuristic Search Planning (HSP) and Fast-Forward (FF) seek to define heuristics by solving relaxed versions of the problem, which includes removing preconditions or negative effects of actions. The de facto standard for state-space search today is a variant of these methods named Fast Downward, whose heuristic relies on building a causal graph to decompose the problem hierarchically — effectively taking the computational hit up front to transform the problem into a handful of approximate but smaller problems that proves itself worthwhile in practice.

Below is an example using pyrobosim and the PDDLStream planning framework, which specifies a more complex goal that uses the predicates we described above. Compared to our example in this post, the video below has a few subtle differences in that there is a separate class of objects to represent the rooms that the robot can navigate, and locations can have multiple placement areas (for example, the counter has separate left and right areas).

• (At robot bedroom)
• (At apple0 table0_tabletop)
• (At banana0 counter0_left)
• (Holding robot water0)

pyrobosim example showing a plan that sequences navigation, pick, and place actions.

Alternative search methods

While forward state-space search is one of the most common ways to plan, it is not the only one. There are many alternative search methods that seek to build alternate data structures whose goal is to avoid the computational complexity of expanding a full state-transition model as above. Some common methods and terms you will see include:

• Plan-space planning (PSP): see these slides for a good example
• Planning graphs: most notably the Graphplan algorithm
• Partial-order planning: most notably the Partial Order Planning Forwards (POPF) algorithm.

In general, these search methods tend to outperform state-space search in tasks where there are several actions that can be performed in any order relative to each other because they depend on, and affect, mutually exclusive parts of the world. One of the canonical simple examples is the “dinner date example” which is used to demonstrate Graphplan. While these slides describe the problem in more detail, the idea is that a person is preparing a dinner date where the end goal is that dinner and a present be ready while the house is clean — or (dinner present ¬garb). By thinking about the problem in this planning graph structure, we can gain the following insights:

1. Planning seeks to find actions that can be executed independently because they are mutually exclusive (mutex). This where the notion of partial ordering comes in, and why there are computational gains compared to explicit state-space search: Instead of separately searching through alternate paths where one action occurs before the other, here we simply say that the actions are mutex and we can figure out which to execute first after planning is complete.
2. This problem cannot be solved at one level because cooking requires clean hands (cleanH) and wrapping the present requires quiet, whereas the two methods of taking out the garbage (carry and dolly) negate these predicates, respectively. So in the solution below, we must expand two levels of the planning graph and then backtrack to get a concrete plan. We first execute cook to remove the requirement on cleanH, and then perform the other two actions in any order — it does not matter.
3. There is an alternative solution where at the first level we execute the wrap action, and at the second level we can execute cook and dolly (in either order) to achieve the goal. You can imagine this solution would be favored if we additionally required our dinner date hero to have clean hands before starting their date — gross!

Planning graph for the dinner date problem [Source].
Straight lines are preconditions and effects, lines with boxes are no-operation (no-op) links, and curved lines are mutual exclusion (mutex) links.
One solution to the problem is highlighted in blue by backtracking from the goal state.

Towards more complex tasks

Let’s get back to our mobile robot example. What if we want to describe more complex action preconditions and/or goal specifications? After all, we established at the start of this post that simple tasks probably don’t need the giant hammer that is a task planner. For example, instead of requesting that a specific object be placed at a specific location, we could move towards something more general like “all apples should be on the table”.

New example world containing objects, apple0 and apple1, which belong to the same object type (apple).

To explore this, let’s add predicates denoting objects belonging to categories. For example, (Type ?t) can define a specific type and (Is ?o ?t) to denote that an object belongs to such a type. Concretely, we could say that (Type apple), (Is apple0 apple), and (Is apple1 apple).

We can then add a new derived predicate (Has ?loc ?entity) to complete this. Derived predicates are just syntactic sugar — in this case, it helps us to succinctly define an entire space of possible states from our library of existing predicates. However, it lets us express more elaborate ideas in less text. For example:

• (Has table apple0) is true if the object apple0 is at the location table.
• (Has table apple) is true if at least one object of type apple is at the location table.

If we choose the goal state (Has table apple) for our example, a solution could involve placing either apple0 or apple1 on the table. One implementation of this derived predicate could look as follows, which makes use of the exists keyword in PDDL.

(:derived (Has ?loc ?entity)
(or
; CASE 1: Entity is a specific object instance, e.g.,
; (Has table apple0)
(and (Location ?loc)
(Obj ?entity)
(At ?entity ?loc)
)
; CASE 2: Entity is a specific object category, e.g.,
; (Has table apple)
(exists (?o)
(and (Obj ?o)
(Location ?loc)
(Type ?entity)
(Is ?o ?entity)
(At ?o ?loc)
)
)
)
)

Another example would be a HasAll derived predicate, which is true if and only if a particular location contains every object of a specific category. In other words, if you planned for (HasAll table apple), you would need both apple0 and apple1 to be relocated. This could look as follows, this time making use of the imply keyword in PDDL.

(:derived (HasAll ?loc ?objtype)
(imply
(and (Obj ?o)
(Type ?objtype)
(Is ?o ?objtype))
(and (Location ?loc)
(At ?o ?loc))
)
)

You could imagine also using derived predicates as preconditions for actions, to similarly add abstraction in other parts of the planning pipeline. Imagine a robot that can take out your trash and recycling. A derived predicate could be useful to allow dumping a container into recycling only if all the objects inside the container are made of recyclable material; if there is any non-recyclable object, then the robot can either pick out the violating object(s) or simply toss the container in the trash.

Below is an example from pyrobosim where we command our robot to achieve the following goal, which now uses derived predicates to express locations and objects either as specific instance, or through their types:

• (Has desk0_desktop banana0)
• (Has counter apple1)
• (HasNone bathroom banana)
• (HasAll table water)

pyrobosim example showing task planning with derived predicates.

Conclusion

This post barely scratched the surface of task planning, but also introduced several resources where you can find out more. One central resource I would recommend is the Automated Planning and Acting textbook by Malik Ghallab, Dana Nau, and Paolo Traverso.

Some key takeaways are:

1. Task planning requires a good model and modeling language to let us describe a task-agnostic planning framework that can be applied to several tasks.
2. Task planning is search, where the output is a plan, or a sequence of actions.
3. State-space search relies on clever heuristics to work in practice, but there are alternative partial-order methods that aim to make search tractable in a different way.
4. PDDL is one of the most common languages used in task planning, where a domain is defined as the tool to solve several problems comprising a set of objects, initial state, and goal specification.
5. PDDL is not the only way to formalize task planning. Here are a few papers I enjoyed:
   • Task Planning in Robotics: an Empirical Comparison of PDDL-based and ASP-based Systems by Jiang et al. (2018) discusses Answer Set Programming (ASP) in addition to PDDL, but also is a fantastic introduction to task planning in general.
   • Towards Manipulation Planning with Temporal Logic Specifications by He et al. (2015) introduces problems similar to our examples, but search is based on automata synthesized from Linear Temporal Logic (LTL) specifications.

After reading this post, the following should now make a little more sense: Some of the more recent task planning systems geared towards robotics, such as the ROS2 Planning System (PlanSys2) and PDDLStream, leverage some of these planners mentioned above. Specifically, these use Fast Downward and POPF.

One key pitfall of task planning — at least in how we’ve seen it so far — is that we are operating at a level of abstraction that is not necessarily ideal for real-world tasks. Let’s consider these hypothetical scenarios:

• If a plan involves navigating between two rooms, but the rooms are not connected or the robot is too large to pass through, how we do know this before we’re actually executing the plan?
• If a plan involves placing 3 objects on a table, but the objects physically will not fit on that table, how do we account for this? And what if there is some other combination of objects that does fit and does satisfy our goal?

To address these scenarios, there possibly is a way to introduce more expert knowledge to our domain as new predicates to use in action preconditions. Another is to consider a richer space of action parameters that reasons about metric details (e.g., specific poses or paths to a goal) more so than a purely symbolic domain would. In both cases, we are thinking more about action feasibility in task planning. This leads us to the field commonly known as task and motion planning (TAMP), and will be the focus of the next post — so stay tuned!

In the meantime, if you want to explore the mobile robot examples in this post, check out my pyrobosim repository and take a look at the task and motion planning documentation.

You can read the original article at Roboticseabass.com.

• Part 1: Getting started with ROS2
• Part 2: Porting robot_controllers
• Part 3: Porting the head camera

ROS2 Humble

The latest ROS2 release came out just a few weeks ago. ROS2 Humble targets Ubuntu 22.04 and is also a long term support (LTS) release, meaning that both the underlying Ubuntu operating system and the ROS2 release get a full 5 years of support.

Since installing operating systems on robots is often a pain, I only use the LTS releases and so I had to migrate from the previous LTS, ROS2 Foxy (on Ubuntu 20.04). Overall, there aren’t many changes to the low-level ROS2 APIs as things are getting more stable and mature. For some higher level packages, such as MoveIt2 and Navigation2, the story is a bit different.

Visualization

One of the nice things about the ROS2 Foxy release was that it targeted the same operating system as the final ROS1 release, Noetic. This allowed users to have both ROS1 and ROS2 installed side-by-side. If you’re still developing in ROS1, that means you probably don’t want to upgrade all your computers quite yet. While my robot now runs Ubuntu 22.04, my desktop is still running 18.04.

Therefore, I had to find a way to visualize ROS2 data on a computer that did not have the latest ROS2 installed. Initially I tried the Foxglove Studio, but didn’t have any luck with things actually connecting using the native ROS2 interface (the rosbridge-based interface did work). Foxglove is certainly interesting, but so far it’s not really an RVIZ replacement – they appear to be more focused on offline data visualization.

I then moved onto running rviz2 inside a docker environment – which works well when using the rocker tool:

sudo apt-get install python3-rocker
sudo rocker --net=host --x11 osrf/ros:humble-desktop rviz2


If you are using an NVIDIA card, you’ll need to add --nvidia along with --x11.

In order to properly visualize and interact with my UBR-1 robot, I needed to add the ubr1_description package to my workspace in order to get the meshes and also my rviz configurations. To accomplish this, I needed to create my own docker image. I largely based it off the underlying ROS docker images:

ARG WORKSPACE=/opt/workspace

ENTRYPOINT ["/ros_entrypoint.sh"]
CMD ["bash"]


The image derives from humble-desktop and then adds the build tools and clones my repository. I then ignore the majority of packages, install dependencies and then build the workspace. The ros_entrypoint.sh script handles sourcing the workspace configuration.

#!/bin/bash
set -e

# setup ros2 environment
source "/opt/workspace/install/setup.bash"
exec "$@"


I could then create the docker image and run rviz inside it:

docker build -t ubr:main
sudo rocker --net=host --x11 ubr:main rviz2


The full source of these docker configs is in the docker folder of my ubr_reloaded repository. NOTE: The updated code in the repository also adds a late-breaking change to use CycloneDDS as I’ve had numerous connectivity issues with FastDDS that I have not been able to debug.

Visualization on MacOSX

I also frequently want to be able to interact with my robot from my Macbook. While I previously installed ROS2 Foxy on my Intel-based Macbook, the situation is quite changed now with MacOSX being downgraded to Tier 3 support and the new Apple M1 silicon (and Apple’s various other locking mechanisms) making it harder and harder to setup ROS2 directly on the Macbook.

As with the Linux desktop, I tried out Foxglove – however it is a bit limited on Mac. The MacOSX environment does not allow opening the required ports, so the direct ROS2 topic streaming does not work and you have to use rosbridge. I found I was able to visualize certain topics, but that switching between topics frequently broke.

At this point, I was about to give up, until I noticed that Ubuntu 22.04 arm64 is a Tier 1 platform for ROS2 Humble. I proceeded to install the arm64 version of Ubuntu inside Parallels (Note: I was cheap and initially tried to use the VMWare technology preview, but was unable to get the installer to even boot). There are a few tricks here as there is no arm64 desktop installer, so you have to install the server edition and then upgrade it to a desktop. There is a detailed description of this workflow on askubuntu.com. Installing ros-humble-desktop from arm64 Debians was perfectly easy.

rviz2 runs relatively quick inside the Parallels VM, but overall it was not quite as quick or stable as using rocker on Ubuntu. However, it is really nice to be able to do some ROS2 development when traveling with only my Macbook.

Migration Notes

Note: each of the links in this section is to a commit or PR that implements the discussed changes.

In the core ROS API, there are only a handful of changes – and most of them are actually simply fixing potential bugs. The logging macros have been updated for security purposes and require c-strings like the old ROS1 macros did. Additionally the macros are now better at detecting invalid substitution strings. Ament has also gotten better at detecting missing dependencies. The updates I made to robot_controllers show just how many bugs were caught by this more strict checking.

image_pipeline has had some minor updates since Foxy, mainly to improve consistency between plugins and so I needed to update some topic remappings.

Navigation has the most updates. amcl model type names have been changed since the models are now plugins. The API of costmap layers has changed significantly, and so a number of updates were required just to get the system started. I then made a more detailed pass through the documentation and found a few more issues and improvements with my config, especially around the behavior tree configuration.

I also decided to do a proper port of graceful_controller to ROS2, starting from the latest ROS1 code since a number of improvements have happened in the past year since I had originally ported to ROS2.

Next steps

There are still a number of new features to explore with Navigation2, but my immediate focus is going to shift towards getting MoveIt2 setup on the robot, since I can’t easily swap between ROS1 and ROS2 anymore after upgrading the operating system.

I believe that one of the best ways to get the training you need for a job market in robotics is to attend tutorials at conferences like ICRA. Unlike workshops where you might listen to some work-in-progress, other workshop paper presentations and panel discussions, tutorials are exactly what they sound like. They aim to give you some hands-on learning sessions on technical tools/skills with specific learning objectives.

As such, most tutorials would expect you to come prepared to actively participate and follow along. For instance, the “Tools for Robotic Reinforcement Learning” tutorial expects you to come knowing how to code in python and have basic knowledge of reinforcement learning because you’ll be expected to use those skills/knowledge in the hands-on sessions.

There are seven tutorials this year.

Those interested in the intersection of machine learning and robotics (yes, yes, robotics and AI don’t refer to the same things, even if many people think they are the same) might find the following tutorials interesting.

• Tools for Robotic Reinforcement Learning
• Data-driven revolution in autonomous driving
• Learning Motion Control for Mobile Robot Navigation – A Tutorial

For those who are more into robots that navigate around our environment, the NavAbility Tutorial Workshop on Non-Gaussian SLAM and Computation is a series of hands-on tutorials that would be highly interesting for you. Be prepared to come with your own laptop to go from getting to know non-gaussian SLAM to solve your SLAM problems.

Meanwhile, roboticists who are more hardware and design-oriented might find the Jamming in Robotics: From Fundamental Building Blocks to Robotic Applications tutorial useful. By “jamming”, they don’t mean musicians coming together to create cool music together — I only found this out through the tutorial website. It refers to the way robots can grab items without needing to have traditional, fingered grippers.

The Tutorial on Koopman Operator and Lifting Linearization: Emerging Theory and Applications of Exact Global Linearization would be interesting for anyone interested in mathy/control/theory side of robotics. Koopman operators have been the buzz in the robotics community recently, and the tutorial is sure to give you the in-depth look at what the buzz is all about.

Lastly, the How to write an R-article and benchmark your results tutorial is one to watch for. It will tell you all about publishing reproducibility-friendly articles, and emphasize the usefulness of doing research in reproducible ways.

Source: https://www.subt-explorer.com/post/fall-2020-update

This is a new series looking at the detailed design of various robots. To start with we will be looking at the design of two different robots that were used for the DARPA Subterranean Challenge. Both of these robots were designed for operating in complex subterranean environments, including Caves, Mines & Urban environments. Both of these robots presented are from the Carnegie Mellon University Explorer team. While I am writing these posts, this was a team effort that required many people to be successful. (If anyone on Team Explorer is reading this, thank you for everything, you are all awesome.)

These posts are skipping the system requirements step of the design process. See here for more details on defining system requirements.

SubT Ground UGV and DS Drone ImageTeam Explorer R1 Ground Robot and DS Drone [Source]

R3 Ground robot (UGV)

For the SubT challenge three ground vehicles were developed all of a similiar design. The ground robots were known with the moniker of R#, where # is the order we built them in. The primary difference between the three versions are

R1 – Static Chassis, so the chassis has minimal ground compliance when driving over obstacles and uneven surfaces. R1 was initially supposed to have a differencing mechanism for compliance, however due to time constraints it was left out from this first version. R1 is pictured above.

R2 – Has the differencing mechanism and was designed as initially planned.

R3 – Is almost identical to R2, but smaller. This robot was built for navigating smaller areas and also to be able to climb up and down steps. It also uses different motors for the driving the wheels.

DS drone

The original drone design used by Team Explorer called their drones D1, D2, etc.. This let a combination of UGV +Drone go by joint designations such as R2D2. Early on, the team switched to a smaller drone design that was referred to as DS1, DS2, etc.. Where DS is short for Drone Small.

The drone design post are split into two sections. The first is about the actual drone platform, and the second is about the payload that sat on top of the drone.

Mechanical & wheels

Robot size decision

After we have the list of system requirements we start with the design of the mechanical structure of the robot. In this case we decided that a wheeled robot would be best. We wanted to have the largest wheels possible to help climb over obstacles, however, we also needed to keep our sensors at the top of the vehicle above the wheels and be able to fit in openings 1 x 1 meters. These requirements set the maximum size of the robot as well as the maximum size of the wheels.

The final dimensions of the first two vehicles (R1 and R2) were around (L x W x H) 1.2 x 0.8 x 0.8 meters (3.9 x 2.6 x 2.6 ft). The third smaller vehicle was around 1 x 0.6 m (3.2 x 1.9 ft) and designed to fit through 0.7×0.7 m openings.

Steering approach

Early on we also needed to determine the method of driving. Do we want wheels or tracks? Do we want to steer with ackerman steering, rocker-bogie, skid steer, etc.?

See here for more details on steering selection.

We chose to use a skid steer four wheeled drive approach for the simplicity of control and the ability to turn in place (point turns). At the start of the competition we were not focused on stair climbing, which might have changed some of our design decisions.

Suspension

The next step was to determine the suspension type. A suspension is needed so that all four of the wheels make contact with the ground. If the robot had a static fixed frame only three of the wheels might make contact with the ground when on uneven surfaces. This would reduce our stability and traction.

We decided early on that we wanted a passive suspension for the simplicity of not having active components. With a passive suspension we were looking at different type of body averaging. We roughly had two choices, front-pivot or side-to-side.

Left image shows a front-pivot approach. Right image shows a side-to-side differencing method.

We decided to choose the front-pivot method, however we decided to make the pivot be roughly centered in the vehicle. This allowed us to put all of the electronics in the front and the batteries in the rear. The front-pivot method we felt would be better for climbing up stairs and for climbing over obstacles on level’ish terrain. Also importantly this approach made it easier to carry a drone on the ground vehicle.

Chassis design

At this point we started designing the chassis. This was an important step so that we could estimate the total weight in order to spec the drive-train. Ideas for the chassis were everything from building with 80/20 to building an aluminum frame and populating it with components, to a solid welded chassis. We selected to use a welded steel tube chassis for the strength. We needed a robot that could survive anything we did to it. This proved to be a wise decision when the robot crashed or fell over cliffs. The downside of the steel was increased mass.

For the pivot we found a large crossed roller bearing that we were able to use to attach the two steel boxes together. The large bore in the middle was useful for passing wires/cables through for batteries, motors, etc…

Part of the chassis design was also determining where all of the components should mount. Having the batteries (green boxes in image above) in the rear helps us climb obstacles. Other goals were to keep the ground clearance as high as possible while keeping the center of gravity (CG) as low as possible. Since those are competing goals, part of the design process was to develop a happy medium.

In order to maintain modularity for service, each wheel module had the motor controller, motor, gear box, and bearing block as a solid unit that could be swapped between robots if there was any issues. This also allowed most of the wiring to be part of that block. The only cables that needed to be connected to each of the modules from the robot were power, CAN communications, and the emergency stop line; all of which were connectorized.

For electronics on R1 and R2 we built an electronics box that was separate from the robot and could be removed from the robot as needed. On R3 we built the electronics into the robot itself. This modular approach was very useful when we had to do some welding to the chassis post-build for modifications. The downside of the modular approach for electronics was that working in the electronics box was more difficult then in the open R3. Also the time for fabricating and wiring the R1/R2 electronics boxes was considerably more than the open R3 electronics. We also had several failures during testing related to the connectors from the electronics boxes.

Wheel design

We debated a lot about what type of wheel to use, ultimately we used motorcycle wheels due to the simplicity of obtaining them and mounting them. The wheel diameter we desired also lined up very well with motorcycle wheels. In order to get better traction and ability to climb over obstacles we liked the wider tires.

R1 and R2 had a wheel diameter of 0.55m, R3 had a wheel diameter of 0.38m. This gave R1 and R2 a ground clearance of 0.2m, and R3 a ground clearance of 0.12m.

The wheel hubs ended up being a different story. We found solid metal rims that we had to machine large amounts of metal out of in order to balance the strength and the weight.

The R1 and R2 robots were around 180kg (400lb)*, the wheels were for a vehicle significantly heavier. As such we put a small amount of pressure in the wheels to keep them from falling off, however we tried to keep the pressure low to increase the ground compliance of the wheels. This method added a very small amount of compliance, we tried removing some of the rubber from the sidewalls, but was not able to get a happy medium between limiting the wheel deforming during point turns and increasing ground compliance.

We were also concerned how the motorcycle tires would do when point turning and if we would rip the wheels from the rims. To counter this we installed a beadlock system into each of the wheels. The beadlock was a curved segment installed in multiple places to sandwich the tire to the rim. We never had a wheel separate from the rim, so our approach definitely worked, however it was a pain to install.

*R3 was around 90 kg (200 lbs). We tried using different wheels and tracks to get R3 to climb stairs well. However that story is for another post…

The black rims were solid metal that we machined the wedges into in order to lightweight them. The 3 metal posts in those wedges are the beadlock tensioning bolts. You can also see the castle nut and pin that holds the wheel to the axle. This image is from R2, you can see the gap between the front and rear sections of the robot where the pivot is.

Drive-train selection

Now that we had a mass estimate and system requirements for speed and obstacle clearance we can start to spec the drive-train. The other piece of information that we needed and had to discuss with the electrical team was the voltage of the battery. Different bus voltages greatly affects the motors available for a given speed and torque. We decided on a 51.2v nominal bus voltage. This presented a problem since it was very hard to find the speed/torques we wanted at that voltage. We ended up selecting a 400W 1/2HP motor+gearbox from Oriental Motors with a parallel 100:1 gearbox that allows us to drive at a maximum speed of 2.5m/s.

The part numbers of the motors and gearbox on R1 and R2 were BLVM640N-GFS + GFS6G100FR.

The part numbers of the motors and gearbox on the smaller R3 were Maxon EC 90 Flat + GP81A.

Next steps

Now that we know the mechanics of the robot we can start building it. In the next post we will start looking at the electronics and motor controls. While the nature of the blog makes it seem that this design is a serial process, in reality lots of things are happening in parallel. While the mechanical team is designing the chassis, the electrical team is finding the electrical components needed in order for the mechanical person to know what needs mounted.

It is also important to work with the electrical team to figure out wire routing while the chassis is being developed.

Note of the editor: This post has been merged from the posts “Hands On Ground Robot & Drone Design Series” and “Mechanical & Wheels – Hands On Ground Robot Design“.

By Marco Arruda

In this post you’ll learn how to program a robot to avoid obstacles using ROS2 and C++. Up to the end of the post, the Dolly robot moves autonomously in a scene with many obstacles, simulated using Gazebo 11.

You’ll learn:

• How to publish AND subscribe topics in the same ROS2 Node
• How to avoid obstacles
• How to implement your own algorithm in ROS2 and C++

1 – Setup environment – Launch simulation

Before anything else, make sure you have the rosject from the previous post, you can copy it from here.

Launch the simulation in one webshell and in a different tab, checkout the topics we have available. You must get something similar to the image below:

2 – Create the node

In order to have our obstacle avoidance algorithm, let’s create a new executable in the file ~/ros2_ws/src/my_package/obstacle_avoidance.cpp:

#include "geometry_msgs/msg/twist.hpp"    // Twist
#include "rclcpp/rclcpp.hpp"              // ROS Core Libraries
#include "sensor_msgs/msg/laser_scan.hpp" // Laser Scan

using std::placeholders::_1;

class ObstacleAvoidance : public rclcpp::Node {
public:
  ObstacleAvoidance() : Node("ObstacleAvoidance") {

    auto default_qos = rclcpp::QoS(rclcpp::SystemDefaultsQoS());
    subscription_ = this->create_subscription(
        "laser_scan", default_qos,
        std::bind(&ObstacleAvoidance::topic_callback, this, _1));
    publisher_ =
        this->create_publisher("cmd_vel", 10);
  }

private:
  void topic_callback(const sensor_msgs::msg::LaserScan::SharedPtr _msg) {
    // 200 readings, from right to left, from -57 to 57 degress
    // calculate new velocity cmd
    float min = 10;
    for (int i = 0; i < 200; i++) { float current = _msg->ranges[i];
      if (current < min) { min = current; } } 
    auto message = this->calculateVelMsg(min);
    publisher_->publish(message);
  }
  geometry_msgs::msg::Twist calculateVelMsg(float distance) {
    auto msg = geometry_msgs::msg::Twist();
    // logic
    RCLCPP_INFO(this->get_logger(), "Distance is: '%f'", distance);
    if (distance < 1) {
      // turn around
      msg.linear.x = 0;
      msg.angular.z = 0.3;
    } else {
      // go straight ahead
      msg.linear.x = 0.3;
      msg.angular.z = 0;
    }
    return msg;
  }
  rclcpp::Publisher::SharedPtr publisher_;
  rclcpp::Subscription::SharedPtr subscription_;
};

int main(int argc, char *argv[]) {
  rclcpp::init(argc, argv);
  rclcpp::spin(std::make_shared());
  rclcpp::shutdown();
  return 0;
}

In the main function we have:

• Initialize node rclcpp::init
• Keep it running rclcpp::spin

Inside the class constructor:

• Subcribe to the laser scan messages: subscription_
• Publish to the robot diff driver: publisher_

The obstacle avoidance intelligence goes inside the method calculateVelMsg. This is where decisions are made based on the laser readings. Notice that is depends purely on the minimum distance read from the message.

If you want to customize it, for example, consider only the readings in front of the robot, or even check if it is better to turn left or right, this is the place you need to work on! Remember to adjust the parameters, because the way it is, only the minimum value comes to this method.

3 – Compile the node

This executable depends on both geometry_msgs and sensor_msgs, that we have added in the two previous posts of this series. Make sure you have them at the beginning of the ~/ros2_ws/src/my_package/CMakeLists.txt file:

# find dependencies
find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(geometry_msgs REQUIRED)
find_package(sensor_msgs REQUIRED)

And finally, add the executable and install it:

# obstacle avoidance
add_executable(obstacle_avoidance src/obstacle_avoidance.cpp)
ament_target_dependencies(obstacle_avoidance rclcpp sensor_msgs geometry_msgs)

...

install(TARGETS
  reading_laser
  moving_robot
  obstacle_avoidance
  DESTINATION lib/${PROJECT_NAME}/
)

Compile the package:
colcon build --symlink-install --packages-select my_package

4 – Run the node

In order to run, use the following command:
ros2 run my_package obstacle_avoidance

It will not work for this robot! Why is that? We are subscribing and publishing to generic topics: cmd_vel and laser_scan.

We need a launch file to remap these topics, let’s create one at ~/ros2_ws/src/my_package/launch/obstacle_avoidance.launch.py:

from launch import LaunchDescription
from launch_ros.actions import Node

def generate_launch_description():

    obstacle_avoidance = Node(
        package='my_package',
        executable='obstacle_avoidance',
        output='screen',
        remappings=[
            ('laser_scan', '/dolly/laser_scan'),
            ('cmd_vel', '/dolly/cmd_vel'),
        ]
    )

    return LaunchDescription([obstacle_avoidance])

Recompile the package, source the workspace once more and launch it:
colcon build --symlink-install --packages-select my_package
source ~/ros2_ws/install/setup.bash
ros2 launch my_package obstacle_avoidance.launch.py

Related courses & extra links:

• ROS2 Basics for C++
• Copy my rosject here
• Dolly robot – Official repo

The post Exploring ROS2 with a wheeled robot – #4 – Obstacle Avoidance appeared first on The Construct.

In this post you’ll learn how to publish to a ROS2 topic using ROS2 C++. Up to the end of the video, we are moving the robot Dolly robot, simulated using Gazebo 11.

You’ll learn:

• How to create a node with ROS2 and C++
• How to public to a topic with ROS2 and C++

1 – Setup environment – Launch simulation

Before anything else, make sure you have the rosject from the previous post, you can copy it from here.

Launch the simulation in one webshell and in a different tab, checkout the topics we have available. You must get something similar to the image below:

2 – Create a topic publisher

Create a new file to container the publisher node: moving_robot.cpp and paste the following content:

#include <chrono>
#include <functional>
#include <memory>

#include "rclcpp/rclcpp.hpp"
#include "geometry_msgs/msg/twist.hpp"

using namespace std::chrono_literals;

/* This example creates a subclass of Node and uses std::bind() to register a
 * member function as a callback from the timer. */

class MovingRobot : public rclcpp::Node {
public:
  MovingRobot() : Node("moving_robot"), count_(0) {
    publisher_ =
        this->create_publisher("/dolly/cmd_vel", 10);
    timer_ = this->create_wall_timer(
        500ms, std::bind(&MovingRobot::timer_callback, this));
  }

private:
  void timer_callback() {
    auto message = geometry_msgs::msg::Twist();
    message.linear.x = 0.5;
    message.angular.z = 0.3;
    RCLCPP_INFO(this->get_logger(), "Publishing: '%f.2' and %f.2",
                message.linear.x, message.angular.z);
    publisher_->publish(message);
  }
  rclcpp::TimerBase::SharedPtr timer_;
  rclcpp::Publisher::SharedPtr publisher_;
  size_t count_;
};

int main(int argc, char *argv[]) {
  rclcpp::init(argc, argv);
  rclcpp::spin(std::make_shared());
  rclcpp::shutdown();
  return 0;
}QoS (Quality of Service)

Similar to the subscriber it is created a class that inherits Node. A publisher_ is setup and also a callback, although this time is not a callback that receives messages, but a timer_callback called in a frequency defined by the timer_ variable. This callback is used to publish messages to the robot.

The create_publisher method needs two arguments:

• topic name
• QoS (Quality of Service) – This is the policy of data saved in the queue. You can make use of different middlewares or even use some provided by default. We are just setting up a queue of 10. By default, it keeps the last 10 messages sent to the topic.

The message published must be created using the class imported:

message = geometry_msgs::msg::Twist();

We ensure the callback methods on the subscribers side will always recognize the message. This is the way it has to be published by using the publisher method publish.

3 – Compile and run the node

In order to compile we need to adjust some things in the ~/ros2_ws/src/my_package/CMakeLists.txt. So add the following to the file:

• Add the geometry_msgs dependency
• Append the executable moving_robot
• Add install instruction for moving_robot

find_package(geometry_msgs REQUIRED)
...
# moving robot
add_executable(moving_robot src/moving_robot.cpp)
ament_target_dependencies(moving_robot rclcpp geometry_msgs)
...
install(TARGETS
  moving_robot
  reading_laser
  DESTINATION lib/${PROJECT_NAME}/
)

We can run the node like below:

source ~/ros2_ws/install/setup.bash
ros2 run my_package

Related courses & extra links:

• ROS2 Basics for C++
• Copy my rosject here
• Dolly robot – Official repo

The post Exploring ROS2 using wheeled Robot – #3 – Moving the Robot
appeared first on The Construct.

This is the second chapter of the series “Exploring ROS2 with a wheeled robot”. In this episode, you’ll learn how to subscribe to a ROS2 topic using ROS2 C++.

You’ll learn:

• How to create a node with ROS2 and C++
• How to subscribe to a topic with ROS2 and C++
• How to launch a ROS2 node using a launch file

1 – Setup environment – Launch simulation

Before anything else, make sure you have the rosject from the previous post, you can copy it from here.

Launch the simulation in one webshell and in a different tab, checkout the topics we have available. You must get something similar to the image below:

2 – Create a ROS2 node

Our goal is to read the laser data, so create a new file called reading_laser.cpp:

touch ~/ros2_ws/src/my_package/reading_laser.cpp

And paste the content below:

#include "rclcpp/rclcpp.hpp"
#include "sensor_msgs/msg/laser_scan.hpp"

using std::placeholders::_1;

class ReadingLaser : public rclcpphttps://www.theconstructsim.com/exploring-ros2-with-wheeled-robot-2-how-to-subscribe-to-ros2-laser-scan-topic/::Node {

public:
  ReadingLaser() : Node("reading_laser") {

    auto default_qos = rclcpp::QoS(rclcpp::SystemDefaultsQoS());

    subscription_ = this->create_subscription(
        "laser_scan", default_qos,
        std::bind(&ReadingLaser::topic_callback, this, _1));
  }

private:
  void topic_callback(const sensor_msgs::msg::LaserScan::SharedPtr _msg) {
    RCLCPP_INFO(this->get_logger(), "I heard: '%f' '%f'", _msg->ranges[0],
                _msg->ranges[100]);
  }
  rclcpp::Subscription::SharedPtr subscription_;
};

int main(int argc, char *argv[]) {
  rclcpp::init(argc, argv);
  auto node = std::make_shared();
  RCLCPP_INFO(node->get_logger(), "Hello my friends");
  rclcpp::spin(node);
  rclcpp::shutdown();
  return 0;
}

We are creating a new class ReadingLaser that represents the node (it inherits rclcpp::Node). The most important about that class are the subscriber attribute and the method callback. In the main function we are initializing the node and keep it alive (spin) while its ROS connection is valid.

The subscriber constructor expects to get a QoS, that stands for the middleware used for the quality of service. You can have more information about it in the reference attached, but in this post we are just using the default QoS provided. Keep in mind the following parameters:

• topic name
• callback method

The callback method needs to be binded, which means it will not be execute at the subscriber declaration, but when the callback is called. So we pass the reference of the method and setup the this reference for the current object to be used as callback, afterall the method itself is a generic implementationhttps://www.theconstructsim.com/exploring-ros2-with-wheeled-robot-2-how-to-subscribe-to-ros2-laser-scan-topic/ of a class.

3 – Compile and run

In order to compile the cpp file, we must add some instructions to the ~/ros2_ws/src/my_package/src/CMakeLists.txt:

• Look for find dependencies and include the sensor_msgs library
• Just before the install instruction add the executable and target its dependencies
• Append another install instruction for the new executable we’ve just created

# find dependencies
find_package(ament_cmake REQUIRED)
find_package(rclcpp REQUIRED)
find_package(sensor_msgs REQUIRED)
...

...
add_executable(reading_laser src/reading_laser.cpp)
ament_target_dependencies(reading_laser rclcpp std_msgs sensor_msgs)
...

...
install(TARGETS
  reading_laser
  DESTINATION lib/${PROJECT_NAME}/
)

Compile it:

colcon build --symlink-install --packages-select my_package

4 – Run the node and mapping the topic

In order to run the executable created, you can use:

ros2 run my_package reading_laser

Although the the laser values won’t show up. That’s because we have a “hard coded” topic name laser_scan. No problem at all, when we can map topics using launch files. Create a new launch file ~/ros2_ws/src/my_package/launch/reading_laser.py:

from launch import LaunchDescription
from launch_ros.actions import Node

def generate_launch_description():

    reading_laser = Node(
        package='my_package',
        executable='reading_laser',https://www.theconstructsim.com/exploring-ros2-with-wheeled-robot-2-how-to-subscribe-to-ros2-laser-scan-topic/
        output='screen',
        remappings=[
            ('laser_scan', '/dolly/laser_scan')
        ]
    )

    return LaunchDescription([
        reading_laser
    ])

In this launch file there is an instance of a node getting the executable as argument and it is setup the remappings attribute in order to remap from laser_scan to /dolly/laser_scan.

Run the same node using the launch file this time:

ros2 launch my_package reading_laser.launch.py

Add some obstacles to the world and the result must be similar to:

Related courses & extra links:

• ROS2 Basics for C++
• Copy my rosject here
• Dolly robot – Official repo

The post Exploring ROS2 with wheeled robot – #2 – How to subscribe to ROS2 laser scan topic appeared first on The Construct.

This is the first chapter of the series “Exploring ROS2 with a wheeled robot”. In this episode, we setup our first ROS2 simulation using Gazebo 11. From cloning, compiling and creating a package + launch file to start the simulation!

You’ll learn:

• How to Launch a simulation using ROS2
• How to Compile ROS2 packages
• How to Create launch files with ROS2

1 – Start the environment

In this series we are using ROS2 foxy, go to this page, create a new rosject selecting ROS2 Foxy distro and and run it.

2 – Clone and compile the simulation

The first step is to clone the dolly robot package. Open a web shell and execute the following:

cd ~/ros2_ws/src/
git clone https://github.com/chapulina/dolly.git


Source the ROS 2 installation folder and compile the workspace:

source /opt/ros/foxy/setup.bash
cd ~/ros2_ws
colcon build --symlink-install --packages-ignore dolly_ignition


Notice we are ignoring the ignition related package, that’s because we will work only with gazebo simulator.

3 – Create a new package and launch file

In order to launch the simulation, we will create the launch file from the scratch. It goes like:

cd ~/ros2_ws/src
ros2 pkg create my_package --build-type ament_cmake --dependencies rclcpp


After that, you must have the new folder my_package in your workspace. Create a new folder to contain launch files and the new launch file as well:

mkdir -p ~/ros2_ws/src/my_package/launch
touch ~/ros2_ws/src/my_package/launch/dolly.launch.py


Copy and paste the following to the new launch file:

import os

from ament_index_python.packages import get_package_share_directory
from launch import LaunchDescription
from launch.actions import DeclareLaunchArgument
from launch.actions import IncludeLaunchDescription
from launch.launch_description_sources import PythonLaunchDescriptionSource

def generate_launch_description():

    pkg_gazebo_ros = get_package_share_directory('gazebo_ros')
    pkg_dolly_gazebo = get_package_share_directory('dolly_gazebo')

    gazebo = IncludeLaunchDescription(
        PythonLaunchDescriptionSource(
            os.path.join(pkg_gazebo_ros, 'launch', 'gazebo.launch.py')
        )
    )

    return LaunchDescription([
        DeclareLaunchArgument(
            'world',
            default_value=[
                os.path.join(pkg_dolly_gazebo, 'worlds', 'dolly_empty.world'), ''
            ],
            description='SDF world file',
        ),
        gazebo
    ])

Notice that a launch file returns a LaunchDescription that contains nodes or other launch files.

In this case, we have just included another launch file gazebo.launch.py and changed one of its arguments, the one that stands for the world name: world.

The robot, in that case, is included in the world file, so there is no need to have an extra spawn node, for example.

And append to the end of the file ~/ros2_ws/src/my_package/CMakeLists.txt the following instruction to install the new launch file into the ROS 2 environment:

install(DIRECTORY
	launch
	DESTINATION share/${PROJECT_NAME}/
)
ament_package()

4 – Compile and launch the simulation

Use the command below to compile only the created package:

cd ~/ros2_ws/
colcon build --symlink-install --packages-select my_package
source ~/ros2_ws/install/setup.bash
ros2 launch my_package dolly.launch.py


5 – Conclusion

This is how you can launch a simulation in ROS2. It is important to notice that:

• We are using a pre-made simulation: world + robot
• This is how a launch file is created: A python script
• In ROS2, you still have the same freedom of including other files or running executables inside a custom launch file

Related courses & extra links:

• ROS2 Basics for C++
• Copy my rosject here
• Dolly robot – Official repo

The post Exploring ROS2 with wheeled robot – #1 – Launch ROS2 Simulation appeared first on The Construct.

Do you fancy making yourself an industrial robot to enjoy at home? Jeremy Fielding, a passionate fan of mechanical engineering, did. So he built one. Good news is: he’s preparing a series of videos to teach you the whole process from scratch. How much power do you need to run 7 motors at one time? If you lose power, how do you prevent the arm from collapsing on you or dropping the load? How do you keep the cost down? He’s recorded over 100 hours of video, and he’s planning to teach you how he used the servo motors, how you can use them for your projects and how he designed his 7 axis, articulated robot.

Jeremy’s aim (website, YouTube, Twitter, Instagram) is simple: draw people to engineering with amazing projects, inspire them with ideas, then teach them how to do it. And for this video series, he’s also looking for your collaboration. So if you’ve got experience and knowledge on building this type of robots and you’d like to share it, maybe you end up being part of the series!

We’d like to thank Black in Robotics for making it possible for us to discover Jeremy. Here’s the video Jeremy has released to introduce his project:

We had to go find this guy on YouTube because he looked very cool! https://t.co/xomix1TZXo @jeremy_fielding #BlackInRobotics #NationalRoboticsWeek @RoboWeek

— Black in Robotics (@BlackInRobotics) April 9, 2021

Check out Jeremy’s YouTube channel to discover many more instructional videos. You can also support his work on Patreon.

Imagine for a moment that a road is used only for a single car and driver. Everything is smooth and wonderful. Then you wake up from that utopian dream and remember that our road networks have multiple cars of varying sizes, from different manufacturers, each with a driver with unique behaviors behind the wheel. We quickly realize that traffic conventions and rules are in place to avoid complete and utter chaos. We believe with increasing robotic use cases in the public domain as we all do see, a similar parallel reality needs to be realized and we propose that RoMi-H, an open-source robot and infrastructure framework that simplifies cross fleet robot collaboration, is the way to achieve this coming reality!

Even before the onset of COVID-19, the number of robots and automation technologies introduced and tested in the healthcare industry has been skyrocketing. Service robots perform an ever increasing and diverse set of tasks; taking on smaller and more sensitive deliveries in some cases and relying more heavily on shared infrastructure such as elevators (lifts), doors, and passageways. No single robotics or automation provider can supply the breadth of solutions required in a modern healthcare facility and no facility can afford to operate siloed systems requiring dedicated infrastructure and operating unique user interfaces. Therein lies the challenge.

ROMI-H allows robots from different vendors to interact with each other as well physical assets in the hospital like elevators. The robots can even avoid gurneys and people.

First announced in July 2018, Robotic Middleware for Healthcare (RoMi-H) is a unique open-source system built on ROS 2 and simulated using the Gazebo simulator. It allows for uniform communication and monitoring across robot platforms, sensors, and enterprise information systems. A brief explanation of this initiative can be found in WIRED Magazine: As Robots Fill the Workplace, They Must Learn to Get Along. In tomorrow’s reality, interoperability must be front and center for every developer, manufacturer, systems integrator, and end-user.

As we have written before:

We need food-delivery robots from one vendor to communicate with drug-delivery robots from another vendor. We need a unified approach to command and control for all the robots in a facility. We need a reliable way to develop and test multi-vendor systems in software simulation prior to deployment. And for it to succeed we need this critical interoperability infrastructure to be open source.

Under the leadership of Singapore’s Centre for Healthcare Assistive and Robotics Technology (CHART) and with collaborators such as IHiS, Hope Technik, GovTech and other solution providers, Open Robotics has been working since 2018 to develop an open-source software solution. Its goal is to realize the potential of a vendor agnostic and interoperable communication system for heterogeneous robots, sensors, and information systems in the healthcare space. To accelerate its development we are encouraging contributions to the open-source codebase to accelerate the development of a robust and sustainable system.

RVIZ Shows each robot’s planned path as well as keep out areas for other robots with ROMI-H

In order to understand the underlying mechanics of RoMi-H, we encourage you to take a look at Programming Multiple Robots with ROS 2. It is being continuously updated and will provide you with a thorough explanation of ROS 2 — upon which RoMi-H is built — and the core Robotics Middleware Framework (RMF) that serves to power RoMi-H. The book also features a tutorial on how one might build a web application that can interface with RoMi-H to create useful applications for robot operators or user-facing tools for the robotics industry.

RoMi-H is able to apply the same software across the different robotic systems while ROS 2 manages the communication and data routing from machine to machine; allowing for real-time, dependable and high-performance data exchanges via a publish-subscribe pattern. Publishers group their messages into different classes and subscribers receive information from the classes of messages they have indicated an interest in. This allows RMF to provide a common platform for integrating heterogeneous robotic systems.

We see the RoMi-H project as a significant step, encouraging an open and integrated approach to robotics development and digitising healthcare. We are looking forward to receiving feedback and contributions from interested parties.

RMF can take a simple map and translate it into a Gazebo simulation. The entire system is powered by ROS 2.

Learn More

A public webinar that introduced RMF and featured a live demonstration took place on 18 August 2020 in the CHART Lab. The presentations and recordings can be viewed here. At the same time, if you are interested in finding out more and viewing the source code, do check out the ROS 2 book and the following repositories:

ROS 2 Book:

• https://osrf.github.io/ros2multirobotbook

Github Repositories:

• RMF: https://github.com/osrf/rmf_core
• Traffic Editor: https://github.com/osrf/traffic_editor
• Free Fleet Management: https://github.com/osrf/free_fleet
• Demo Worlds: https://github.com/osrf/rmf_demos

We would like to acknowledge the Singapore government for their vision and support to start this ambitious research and development project, “Development of Standardised Robotics Middleware Framework – RMF detailed design and common services, large-scale virtual test farm infrastructure, and simulation modelling”. The project is supported by the Ministry of Health (MOH) and the National Robotics Program (NRP).

Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not reflect the views of the NR2PO, MOH or other parties.

It’s Saturday, it’s the turn of another post of the James Bruton focus series, and it’s Boxing Day in the UK and most of the Commonwealth countries. Even if this holiday has nothing to do with boxing, I didn’t want to miss the opportunity to take it literally and bring you a project in which James teamed up with final year degree students in Computer Games Technology at Portsmouth University to build a robot that fights a human in a Virtual Reality (VR) game.

For this project, the students Michael (Coding & VR Hardware), Stephen (Character Design & Animation), George (Environment Art) and Boyan (Character Design & Animation) designed a VR combat game in which you fight another character. James’ addition was to design a real robot that fights the player, so that when they get hit in the game, they also get hit in real life by the robot. The robot and the player’s costume are tracked using Vive trackers so the VR system knows where to position each of them in the 3D virtual environment. You can see some artwork and more details about the project here and here. Without further ado, here’s James’ video:

Happy holidays!

James Bruton playing his barcode synths

As every other Saturday, I’m bringing you another cool open-source project from James Bruton. Today, how about becoming an experimental musician with your own barcode scanner synthesizer?

I introduced James Bruton in the first post of this focus series on him, where I showed you the Boston Dynamics-inspired open robot dogs projects that consolidated him as one of the top maker on YouTube. As a sort of musician, the barcode synth project I’ve picked for this second post grabbed my attention among the countless videos he’s got on his channel.

To be more specific, the barcode synth consists of two projects. A bit more than a year ago, James showed how to build a four-neck guitar synth with the frets (the place where you put your fingers to play a note on the guitar) being barcodes instead of strings. To play this guitar, you only need a barcode reader connected to an Arduino that converts the data read from the barcodes into a number that represents a MIDI note – which is a digital representation of a musical note based on the MIDI standard. You can then plug the Arduino into a synth or your computer (if you love virtual instruments as much as I do!) to transform the MIDI output into actual sound. Extra features of this guitar included pitch bending and octave shifting buttons. You can access the open-source code of this type of guitar here, and enjoy James’ explanation (and performance) in the following video:

A couple of months ago, James made an improved version of the previous guitar synth. Instead of using the number given by the barcode, for this improved synth he hacked the barcode reader to interpret the barcodes as images so that the output is the raw square wave that it sees. With he help of a Teensy microcontroller to do the digital signal processing and a Raspberry Pi to display barcode images on a screen fitted to a 3D-printed guitar, he could produce a richer range of sounds compared to the previous version. If you want to build you own barcode synth, check out the open-source files and his video (you’ll be impressed to find out how a zebra sounds like!):

Make tech, make music, and stay tuned!

James Bruton with openDog V2

What if you could ride your own giant LEGO electric skateboard, make a synthesizer that you can play with a barcode reader, or build a strong robot dog based on the Boston Dynamics dog robot? Today sees the start of a new series of videos that focuses on James Bruton’s open source robot projects.

James Bruton is a former toy designer, current YouTube maker and general robotics, electrical and mechanical engineer. He has a reputation for building robot dogs and building Iron Man inspired cosplays. He uses 3D printing, CNC and sometimes welding to build all sorts of robotics related creations. Highlights include building Mark Rober’s auto-strike bowling ball and working with Colin Furze to build a life-sized Iron Man Hulkbuster for an official eBay and Marvel promo. He also built a life-sized Bumblebee Transformer for Paramount to promote the release of the Bumblebee movie.

I discovered James’ impressive work in this episode of Ricardo Tellez’s ROS Developers Podcast on The Construct, which I highly recommend. Whether you enjoy getting your hands dirty with CAD files, 3D-printed parts, arduinos, motors and code, or you like learning about the full research & development (R&D) process of a robotics project, you will have loads of hours of fun following this series.

Today I brought one of James’ coolest and most successful open source projects: openDog and its different versions. In James’ own words, “if you want your very own four-legged friend to play fetch with and go on long walks then this is the perfect project for you.” You can access all the CAD files and code here. And without further ado, here’s the full YouTube playlist of the first version of openDog:

James also released another series of videos developing an affordable version of openDog: Mini Robot Dog. This robot is half the size of openDog and its mechanical components and 3D-printed parts are much more cheaper than the former robot without sacrificing compliance. You can see the full development in the playlist below, and access the open source files of version 1 and version 2.

Based on the insight gained through the R&D of openDog, Mini Robot Dog and these test dogs, James built the ultimate robot dog: openDog V2. For this improved version of openDog, he used brushless motors which can be back-driven to increase compliance. And by adding an Inertial Measurements Unit, he improved the balance of the robot. CAD files and code are available here. If you want to find out whether the robot is able to walk, check out the openDog V2 video series:

If you like James Bruton’s project, you can check out his website for more resources, updates and support options. See you in the next post of our focus series!

In a tiny fraction of the time it took us to refine TrotBot’s linkage in our garage, we explored numerous variations of Strider’s linkage and their trade-offs, and were able to significantly improve its performance in these categories:

Gait Efficiency. To be energy efficient, walking gaits should be smooth and not cause the robot to rise and fall – think how much more tiring it is to do lunges than it is to simply walk. The feet should also move at constant speed – try walking while changing how quickly your feet sweep across the ground and your leg muscles will quickly complain! Strider Ver 3 was optimized for both of these features, and it’s the only mechanism that we’ve tested which can walk with a 1:1 gear ratio without the LEGO motors stalling:

Rugged Terrain. Like with TrotBot, we wanted Strider to be able to walk on rugged terrain, otherwise why not just use wheels? But, mechanical walkers are blind and dumb, and they can’t lift their feet to avoid tripping on obstacles. So we designed Strider’s foot-path to mimic how you would walk blindfolded on a path crisscrossed by roots: lift your feet high and keep them high until stepping back down to the ground, like this:

Here’s a rugged terrain test an 8-leg build:

Strength. Walkers stress frames more than wheeled vehicles, so we increased the frame’s rigidity with more triangles. Also, the leverage of long legs can put a tremendous amount of force on the leg’s joints, especially when turning them tank-style. We strengthened Strider’s leg joints by sandwiching them between beams like in the image below, which also reduces how quickly friction wears down the lips of the frictionless pins.

Here’s a test of Strider’s strength with a 10 pound load:

We made these changes while keeping Strider simple – it’s still our easiest walker to build. We also (hopefully) made our building instructions more clear. We learned how bad we are at creating building instructions from Ben’s experience teaching his walking robots class last spring. In the past when users would email us about something confusing in the instructions, we’d simply add more pictures to clarify things. This backfired in Ben’s class, since more build pictures resulted in the students getting more confused, and caused them to skim thru the pictures more quickly, and to make more mistakes. As we learned, less is more!

We also separated the leg and frame instructions to make it easier to build different versions of Strider, and we designed the frame to make it easy to swap between the battery box/IR controller, or the EV3 brick. You can find the instructions here. Here’s an example build with the EV3 brick mounted underneath – but it would probably work with the brick mounted on top as well.

For the bold, we also posted a half dozen new variations of the Strider linkage for you to try building here, along with an online optimizer for the even bolder to create their own versions of the Strider linkage. We’ve also optimized Klann’s and Strandbeest’s linkages for LEGO, and you can find their online optimizers here. (jeez, I guess we’ve been busy)

Here’s a final teaser to encourage you (or your students) to take on the challenging task of replacing wheels with mechanical legs:

Cyberbotics Ltd. is launching https://robotbenchmark.net to allow everyone to program simulated robots online for free.

Robotbenchmark offers a series of robot programming challenges that address various topics across a wide range of difficulty levels, from middle school to PhD. Users don’t need to install any software on their computer, cloud-based 3D robotics simulations run on a web page. They can learn programming by writing Python code to control robot behavior. The performance achieved by users is recorded and displayed online, so that they can challenge their friends and show off their skills at robot programming on social networks. Everything is designed to be extremely easy-to-use, runs on any computer, any web browser, and is totally free of charge.

This project is funded by Cyberbotics Ltd. and the Human Brain Project.

About Cyberbotics Ltd.: Cyberbotics is a Swiss-based company, spin-off from the École Polytechnique Fédérale de Lausanne, specialized in the development of robotics simulation software. It has been developing and selling the Webots software for more than 19 years. Webots is a reference software in robotics simulation being used in more than 1200 companies and universities across the world. Cyberbotics is also involved in industrial and research projects, such as the Human Brain Project.

About the Human Brain Project: The Human Brain Project is a large ten-year scientific research project that aims to build a collaborative ICT-based scientific research infrastructure to allow researchers across the globe to advance knowledge in the fields of neuroscience, computing, neurorobotics, and brain-related medicine. The Project, which started on 1 October 2013, is a European Commission Future and Emerging Technologies Flagship. Based in Geneva, Switzerland, it is coordinated by the École Polytechnique Fédérale de Lausanne and is largely funded by the European Union.

The need for fast, accurate 3D mapping solutions has quickly become a reality for many industries wanting to adopt new technologies in AI and automation. New applications requiring these 3D mapping platforms include surveillance, mining, automated measurement & inspection, construction management & decommissioning, and photo-realistic rendering. Here at Clearpath Robotics, we decided to team up with Mandala Robotics to show how easily you can implement 3D mapping on a Clearpath robot.

3D Mapping Overview

3D mapping on a mobile robot requires Simultaneous Localization and Mapping (SLAM), for which there are many different solutions available. Localization can be achieved by fusing many different types of pose estimates. Pose estimation can be done using combinations of GPS measurements, wheel encoders, inertial measurement units, 2D or 3D scan registration, optical flow, visual feature tracking and others techniques. Mapping can be done simultaneously using the lidars and cameras that are used for scan registration and for visual position tracking, respectively. This allows a mobile robot to track its position while creating a map of the environment. Choosing which SLAM solution to use is highly dependent on the application and the environment to be mapped. Although many 3D SLAM software packages exist and cannot all be discussed here, there are few 3D mapping hardware platforms that offer full end-to-end 3D reconstruction on a mobile platform.

Existing 3D Mapping Platforms

We will briefly highlight some more popular alternatives of commercialized 3D mapping platforms that have one or many lidars, and in some cases optical cameras, for point cloud data collection. It is important to note that there are two ways to collect a 3D point cloud using lidars:

1. Use a 3D lidar which consists of one device with multiple stacked horizontally laser beams
2. Tilt or rotate a 2D lidar to get 3D coverage

Tilting of a 2D lidar typically refers to back-and-forth rotating of the lidar about its horizontal plane, while rotating usually refers to continuous 360 degree rotation of a vertically or horizontally mounted lidar.

Example 3D Mapping Platforms: 1. MultiSense SL (Left) by Carnegie Robotics, 2. 3DLS-K (Middle) by Fraunhofer IAIS Institute, 3. Cartographer Backpack (Right) by Google.

1. MultiSense SL

The MultiSense SL was developed by Carnegie Robotics and provides a compact and lightweight 3D data collection unit for researchers. The unit has a tilting Hokuyo 2D lidar, a stereo camera, LED lights, and is pre-calibrated for the user. This allows for the generation of coloured point clouds. This platform comes with a full software development kit (SDK), open source ROS software, and is the sensor of choice for the DARPA Robotics Challenge for humanoid robots.

2. 3DLS-K

The 3DLS-K is a dual-tilting unit made by Fraunhofer IAIS Institute with the option of using SICK LMS-200 or LMS-291 lidars. Fraunhofer IAIS also offers other configurations with continuously rotating 2D SICK or Hokuyo lidars. These systems allow for the collection of non-coloured point clouds. With the purchase of these units, a full application program interface (API) is available for configuring the system and collecting data.

3. Cartographer Backpack

The Cartographer Backpack is a mapping unit with two static Hokuyo lidars (one horizontal and one vertical) and an on-board computer. Google released cartographer software as an open source library for performing 3D mapping with multiple possible sensor configurations. The Cartographer Backpack is an example of a possible configuration to map with this software. Cartographer allows for integration of multiple 2D lidars, 3D lidars, IMU and cameras, and is also fully supported in ROS. Datasets are also publicly available for those who want to see mapping results in ROS.

Mandala Mapping – System Overview

Thanks to the team at Mandala Robotics, we got our hands on one of their 3D mapping units to try some mapping on our own. This unit consists of a mount for a rotating vertical lidar, a fixed horizontal lidar, as well as an onboard computer with an Nvidia GeForce GTX 1050 Ti GPU. The horizontal lidar allows for the implementation of 2D scan registration as well as 2D mapping and obstacle avoidance. The vertical rotating lidar is used for acquiring the 3D point cloud data. In our implementation, real-time SLAM was performed solely using 3D scan registration (more on this later) specifically programmed for full utilization of the onboard GPU. The software used to implement this mapping can be found on the mandala-mapping github repository.

Scan registration is the process of combining (or stitching) together two subsequent point clouds (either in 2D or 3D) to estimate the change in pose between the scans. This results in motion estimates to be used in SLAM and also allows a new point cloud to be added to an existing in order to build a map. This process is achieved by running iterative closest point (ICP) between the two subsequent scans. ICP performs a closest neighbour search to match all points from the reference scan to a point on the new scan. Subsequently, optimization is performed to find rotation and translation matrices that minimise the distance between the closest neighbours. By iterating this process, the result converges to the true rotation and translation that the robot underwent between the two scans. This is the process that was used for 3D mapping in the following demo.

Mandala Robotics has also released additional examples of GPU computing tasks useful for robotics and SLAM. These examples can be found here.

Mandala Mapping Results

The following video shows some of our results from mapping areas within the Clearpath office, lab and parking lot. The datasets collected for this video can be downloaded here.

The Mandala Mapping software was very easy to get up and running for someone with basic knowledge in ROS. There is one launch file which runs the Husky base software as well as the 3D mapping. Initiating each scan can be done by sending a simple scan request message to the mapping node, or by pressing one button on the joystick used to drive the Husky. Furthermore, with a little more ROS knowledge, it is easy to incorporate autonomy into the 3D mapping. Our forked repository shows how a short C++ script can be written to enable constant scan intervals while navigating in a straight line. Alternatively, one could easily incorporate 2D SLAM such as gmapping together with the move_base package in order to give specific scanning goals within a map.

Why use Mandala Mapping on your robot?

If you are looking for a quick and easy way to collect 3D point clouds, with the versatility to use multiple lidar types, then this system is a great choice. The hardware work involved with setting up the unit is minimal and well documented, and it is preconfigured to work with your Clearpath Husky. Therefore, you can be up and running with ROS in a few days! The mapping is done in real time, with only a little lag time as your point cloud size grows, and it allows you to visualize your map as you drive.

The downside to this system, compared to the MultiSense SL for example, is that you cannot yet get a coloured point cloud since no cameras have been integrated into this system. However, Mandala Robotics is currently in the beta testing stage for a similar system with an additional 360 degree camera. This system uses the Ladybug5 and will allow RGB colour to be mapped to each of the point cloud elements. Keep an eye out for a future Clearpath blogs in case we get our hands on one of these systems! All things considered, the Mandala Mapping kit offers a great alternative to the other units aforementioned and fills many of the gaps in functionality of these systems.

The post Rapid Outdoor/Indoor 3D Mapping with a Husky UGV appeared first on Clearpath Robotics.

We are excited to show off a simulation of a Prius in Mcity using ROS Kinetic and Gazebo 8. ROS enabled the simulation to be developed faster by using existing software and libraries. The vehicle’s throttle, brake, steering, and transmission are controlled by publishing to a ROS topic. All sensor data is published using ROS, and can be visualized with RViz.

We leveraged Gazebo’s capabilities to incorporate existing models and sensors.
The world contains a new model of Mcity and a freeway interchange. There are also models from the gazebo models repository including dumpsters, traffic cones, and a gas station. On the vehicle itself there is a 16 beam lidar on the roof, 8 ultrasonic sensors, 4 cameras, and 2 planar lidar.

The simulation is open source and available at on GitHub at osrf/car_demo. Try it out by installing nvidia-docker and pulling “osrf/car_demo” from Docker Hub. More information about building and running is available in the README in the source repository.

The Robot Academy is a new learning resource from Professor Peter Corke and the Queensland University of Technology (QUT), the team behind the award-winning Introduction to Robotics and Robotic Vision courses. There are over 200 lessons available, all for free.

The lessons were created in 2015 for the Introduction to Robotics and Robotic Vision courses. We describe our approach to creating the original courses in the article, An Innovative Educational Change: Massive Open Online Courses in Robotics and Robotic Vision. The courses were designed for university undergraduate students but many lessons are suitable for anybody, as you can easily see the difficulty rating for each lesson. Below are lessons from inverse kinematics and robot motion.

You can watch the entire masterclass on the Robot Academy website.

Introduction

In this video lecture, we will learn about inverse kinematics, that is, how to compute the robot’s joint angles given the desired pose of their end-effector and knowledge about the dimensions of its links. We will also learn about how to generate paths that lead to a smooth coordinated motion of the end-effector.

Inverse kinematics for a 2-joint robot arm using geometry

In this lesson, we revisit the simple 2-link planar robot and determine the inverse kinematic function using simple geometry and trigonometry.

Inverse kinematics for a 2-joint robot arm using algebra

You can watch the entire masterclass on the Robot Academy website.

If you liked this article, you may also enjoy:

• Smart Grasping System available on ROS Development Studio
• The Robot Academy: Lessons in image formation and 3D vision
• The Robot Academy: The open online robotics education resource 
• Machine Learning with OpenAI Gym on ROS Development Studio
• SICK LMS full LIDAR teardown
• Envisioning the future of robotics
• How to mount an external removable hard drive on a robot

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Would you like to make a robot to grasp something, but you think that is impossible to you just because you can’t buy a robot arm? I’m here to tell that you can definitely achieve this without buying a real robot. Let’s see how:

How is that possible?

The Smart Grasping Sandbox built by Shadow Robotics is now available for everybody on the ROS Development Studio – a system that allows you to create and test your robot programs through simulations using only a web browser.

Wait, what is the Smart Grasping Sandbox?

The Smart Grasping Sandbox is a public simulation for the Shadow’s Smart Grasping System with theUR10 robot from Universal Robots. It allows you to make a robot to grasp something without having to learn everything related to Machine Learning, and being available on the ROS Development Studio, it allows you to test it without the hassle of installing all the requirements.

As Ugo Cupcic said:

I don’t want to have to specify every aspect of a problem — I’d rather the system learn the best way to approach a given problem itself.

Using the Development Environment for Grasping

In order to user the Smart Grasping Sandbox on ROS Development Studio—also known as RDS—just go to http://rds.theconstructsim.com and Sign In. Once logged in, go to the Public Simulations, select the Smart Grasping Sandbox simulation and press the red Launch this simulation button.

After pressing the Launch this simulation, a new screen will appears asking you to select the launch file. Just keep the default main.launch and press the red run button.

After a few seconds, the simulation will be loaded and you will be able interact with the simulation.

Once the simulation is loaded, you can see the instructions about how to control it on the left side.

In the center you have the simulation itself, and on the right side you can see the Integrated Development Environment and the Web Shell that allows you to modify the code and send Linuxcommands to the simulation respectively.

You can modify, the simulation, the control programs or even create your own manipulation control programs, make the grasping system learn using deep learning or else. This is up to you as developer!

You can also see what the robot sees using the online ROS 3D Visualizer (RViz) typing the following command on the Web Shell:

$ rosrun rviz rviz

After running Rviz, you press the red Open ROS Graphic Tools button on the bottom left side of the system and a new tab should appear with Rviz.

Conclusion

Here we have seen how to use a simulation to programme a robot to grasp something.

The simulation uses ROS as the middleware to control the robot.

If you want to learn ROS or master your ROS Skills, we recommend you to give a try to Robot Ignite Academy.

If you are a robot builder and want to have your own simulation available for everybody on RDS, or you want to have a course specifically for your robot, contact us through the email info@theconstructsim.com.

Today we are looking at how to program your NAO Robot for Human Interaction. Watch the video and follow the steps below to get interactive with your robot pal!

1.  Create an animation of movements
   • Right click, select “Create a new box”
   • Select “Timeline”

We want the robot to do 3 things, a five high, a hello, and a goodbye move:

• On the Timeline Edit Box
  • Name: High Five
  • Image: Click “Edit”
    • On the Edit box image
    • Click “Browse” and select a file
  • Click “OK”

Now, as you can remember, if we take it to the box, we can get access to the NAO’s Timeline. Mine now is connected to the PC. So what I’m going to do is I’m going to set some points of the NAO to do some movements. As I have said before on one of my tutorial I’m going to make it set all the joints to the whole body.

We want to make him look like he is doing a high five:

• Right click the arm
  • Click “Stiffen chain on/off”
  • Raise his arm up
• Set joints and keyframe
  • Select “Arms”
• Click “Play” on the Motion Controller

That’s a quite quick high five actually. What we want to do is to actually give him a pause when he got his arm in the air. So, the way we do that is to copy and paste the keyframe.

Grab a few already built in animation, just to show you how much it works:

• Go to Motions>Animations
  • Drag a “Hello” box to the workspace
  • Drag a “Wipe Forehead” box for a goodbye move to the workspace

We want to hear him speak, so:

• Go to Voice
  • Drag a “Speech Recognition” box to the workspace
• Drag a “Switch Case”
  • Connect the “Speech Recognition” box to the “Switch Case” box
  • On the first case, write “Hello” and connect to the “Hello” box
  • On the second case, write “High Five” and connect it to the “High Five” box
  • On the third case, write “Goodbye” and connect it to the “Wipe Forehand” box
• Set parameters of Speech Recognition
  • Word list: Hi; High Five; Goodbye;
  • Play with the threshold to see what works better
  • Click “OK”

So there we have a speech recognition, he will listen to what I am saying and then we will act a motion. Now, what we need to do is to make sure that the robot doesn’t repeat it, so you know, it doesn’t loop back. Before we do that, we are going to:

• Drag 3 “Say” box to the workspace so he can  speak back to us
  • On the “Switch case”, connect the “Hello” case on the “Say” box
    • Localized Text
      • Change the language to English(United States)
      • Write “Hi” on the textbox
  • Connect the “High Five” case on the “Say (1)” box
    • Localized Text
      • Change the language to English(United States)
      • Write “High Five” on the textbox
  • Connect the “Goodbye” case on the “Say(2)” box
    • Localized Text
      • Change the language to English(United States)
      • Write “Goodbye” on the textbox

So, here’s what we are saying, he recognizes it, says a word and does an animation, now what we want to do is we don’t want him to loop, as in he keeps hearing the same sound and maybe he starts to hear himself and repeat himself, so to stop that we need to connect the end of the motion boxes back to the “Speech Recognition” box. A simple output to the input speech.

I am going to make a little change actually, instead of “Hello”, we should put “Hi” here and I’m going to put “Hi” in there which is already done because they have to match otherwise if it doesn’t match he won’t recognize it. So if he hit “Load”

Philip: High Five!

Nao: High Five!

Philip: Goodbye!

Nao: Goodbye!

Philip: Hi!

Nao: Hello!

Brilliant! So that looks likes he’s working very well, so again if we says “Hi”, “High Five” or “Good bye” then he will greet you. If you walk in to room and you say “Hi” to a robot, obviously, he will do his animation and say a speech. Here you can use your imagination and try different things, different animations.

If you liked this article, you may also enjoy these from Robo-Phil:

• Python programming your NAO robot
• Hard at work: A review of the Laevo Exoskeleton

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

You probably know the Sphero robot. It is a small robot with the shape of a ball. In case that you have one, you must know that it is possible to control it using ROS, by installing in your computer the Sphero ROS packages developed by Melonee Wise and connecting to the robot using the bluetooth of the computer.

Now, you can use the ROS Development Studio to create ROS control programs for that robot, testing as you go by using the integrated simulation.

The ROS Development Studio (RDS) provides off-the-shelf a simulation of Sphero with a maze environment. The simulation provides the same interface as the ROS module created by Melonee, so you can test your develop and test the programs on the environment, and once working properly, transfer it to the real robot.

We created the simulation to teach ROS to the students of the Robot Ignite Academy. They have to learn ROS enough to make the Sphero get out of the maze by using odometry and IMU.

Using the simulation

To use the Sphero simulation on RDS go to rds.theconstructsim.com and sign in. If you select the Public simulations, you will quickly identify the Sphero simulation.

Press the red Play button. A new screen will appear giving you details about the simulation and asking you which launch file you want to launch. The main.launch selected by default is the correct one, so just press Run.

After a few seconds the simulation will appear together with the development environment for creating the programs for Sphero and testing them.

On the left hand side you have a notebook containing information about the robot and how to program it with ROS. This notebook contains just some examples, but it can be completed and/or modified at your will. As you can see it is an iPython notebook and follows its standard. So it is up to you to modify it, add new information or else. Remember that any change you do to the notebook will be saved in a simulation in your private area of RDS, so you can come back later and launch it with your modifications.

You must know that the code included in the notebook is directly executable by selecting the cell of the code (do a single click on it) and pressing the small play button at the top of the notebook. This means that, once you press that button, the code will be executed and start controlling the Sphero simulated robot for a few time-steps (remember to have the simulation activated (Play button of the simulation activated) to see the robot move).

On the center area, you can see the IDE. It is the development environment for developing the code. You can browse there all the packages related to the simulation or any other packages that you may create.

On the right hand side, you can see the simulation and beneath it, the shell. The simulation shows the Sphero robot as well as the environment of the maze. On the shell, you can issue commands in the computer that contains the simulation of the robot. For instance, you can use the shell to launch the keyboard controller and move the Sphero around. Try typing the following:

• $ roslaunch sphero_gazebo keyboard_teleop.launch

Now you must be able to move the robot around the maze by pressing some keys of the keyboard (instructions provided on the screen).

You can also launch there Rviz, and then watch the robot, the frames and any other additional information you may want of the robot. Type the following:

• $ rosrun rviz rviz

Then press the Screen red icon located at the bottom-left of the screen (named the graphical tools). A new tab should appear, showing how the Rviz is loading. After a while, you can configure the Rviz to show the information you desire.

There are many ways you can configure the screen to provide more focus to what interests you the most.

To end this post, I would like to indicate that you can download the simulation to your computer at any time, by doing right-click on the directories and selecting Download. You can also clone the The Construct simulations repository to download it (among other simulations available).

If you liked this tutorial, you may also enjoy these:

• UgCS photogrammetry technique for UAV land surveying missions
• Programming for robotics: Introduction to ROS
• Python programming your NAO robot
• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Figure 15: Adding 40m overshot to both ends of each survey line

UgCS is easy-to-use software for planning and flying UAV drone-survey missions. It supports almost any UAV platform, providing convenient tools for areal and linear surveys and enabling direct drone control. What’s more, UgCS enables professional land survey mission planning using photogrammetry techniques.

How to plan photogrammetry mission with UgCS

Standard land surveying photogrammetry mission planning with UgCS can be divided into following steps :

1. Obtain input data
2. Plan mission
3. Deploy ground control points
4. Fly mission
5. Image geotagging
6. Data processing
7. Map import to UgCS (optional)

https://www.youtube.com/watch?v=SalCQoG6WUw&feature=youtu.be

Step one: Obtain input data

Firstly, to reach the desired result, input settings have to be defined:

• Required GSD (ground sampling distance – size of single pixel on ground),
• Survey area boundaries,
• Required forward and side overlap.

GSD and area boundaries are usually defined by the customer’s requirements for output material parameters, for example by scale and resolution of digital map. Overlap should be chosen according to specific conditions of surveying area and requirements of data processing software.

Each data processing software (e.g., Pix4D, Agisoft Photoscan, Dronedeploy, Acute 3d) has specific requirements for side and forward overlaps for different surfaces. To choose correct values, please refer to documentation of chosen software. In general, 75% forward and 60% side overlap will be a good choice. Overlapping should be increased for areas with small amount of visual cues, for example for deserts or forests.

Often, aerial photogrammetry beginners are excited about the option to produce digital maps with extremely high resolution (1-2cm/pixel), and to use very small GSD for mission planning. This is very bad practice. Small GSD will result in longer flight time, hundreds of photos for each acre, tens of hours of processing and heavy output files. GSD should be set according to the output requirements of the digital map.

Other limitations can occur. For example, GSD of 10cm/pixel is required, but designed to use a Sony A6000 camera. Based on mentioned GSD and camera’s parameters, the flight altitude would be set to 510 meters. In most countries, maximum allowed altitude of UAV’s (without special permission) is limited to 120m/400ft AGL (above ground). Taking into account the maximum allowed altitude, the maximum possible GSD in this case could be no more than 2.3cm.

Step two: Plan your mission

Mission planning consists of two stages:

• Initial planning,
• Route optimisation.

-Initial planning:

The first step is to set surveying area using UgCS Photogrammetry tool. Area can be set using visual cues on underlying map or using exact coordinates of edges. The result – survey area is marked with yellow boundaries (Figure 1).

Figure 1: Setting the survey area

The next step is to set GSD and overlapping for the camera in Photogrammetry tool’s settings window (Figure 2).

Figure 2: Setting camera’s Ground Sampling Distance and overlapping

To take photos in Photogrammetry tool’s setting window, define the control action of the camera (Figure 3). Set camera by distance triggering action with default values.

Figure 3: Setting camera’s control action

At this point, initial route planning is completed. UgCS will automatically calculate photogrammetry route (see Figure 4).

Figure 4: Calculated photogrammetry survey route before optimisation

-Route optimisation

To optimise the route, it’s calculated parameters should be known: altitude, estimated flight time, number of shots, etc.

Part of the route’s calculated information can be found in the elevation profile window. To access the elevation profile window (if it is not visible on screen) click the parameters icon on the route card (lower-right corner, see Figure 5), and from the drop-down menu select show elevation:

Figure 5: Accessing elevation window from Route cards Parameters settings

The elevation profile window will present an estimated route length, duration, waypoint count and min/max altitude data:

Figure 6: Route values in elevation profile window

To get other calculated values, open route log by clicking on route status indicator: the green check-mark (upper-right corner, see Figure 7) of the route card:

Figure 7: Route card and status indicator, Route log

Using route parameters, it can be optimised to be more efficient and safe.

-Survey line direction

By default, UgCS will trace survey lines from south to north. But, in most cases, it will be more optimal to fly parallel to the longest boundary line of the survey area. To change survey line direction, edit direction angle field in the photogrammetry tool. In the example, by changing angle to 135 degrees, the number of passes is reduced from five (Figure 4) to four (Figure 8) and route length is 1km instead of 1.3km.

Figure 8: Changed survey line angle to be parallel to longest boundary

-Altitude type

UgCS Photogrammetry tool has the option to define how to trace the route according to altitude, with constant altitude above ground (AGL) or above mean sea level (AMSL). Please refer to your data processing software requirements as to which altitude tracking method it recommend.

In the UgCS team’s experience, the choice of altitude type depends on desired result. For orthophotomap (standard aerial land survey output format) it is better to choose AGL to ensure constant GSD for the entire map. If the aim is to produce DEM or 3D reconstruction, use AMSL so the data processing software has more data to correctly determine ground elevation by photos in order to provide more qualitative output.

Figure 9: Elevation profile with constant altitude above mean sea level (AMSL)

In this case, UgCS will calculate flight altitude based on the lowest point of the survey area.

If AGL is selected in photogrammetry tool’s settings, UgCS will calculate the altitude for each waypoint. But in this case, terrain following will be rough if no “additional waypoints” are added (see Figure 10).

Figure 10: Elevation profile with AGL without additional waypoints

Therefore, if AGL is used, add some “additional waypoints” flags and UgCS will calculate a flight plan with elevation profile accordingly (see Figure 11).

Figure 11: Elevation profile with AGL with additional waypoints

-Speed

In general, if flight speed is increased it will minimise flight time. But high speed in combination with large camera exposure can result in blurred images. In most cases 10m/s is the best choice.

-Camera control method

UgCS supports 3 camera control methods (actions):

1. Make a shot (trigger camera) in waypoint,
2. Make shot every N seconds,
3. Make shot every N meters.

Not all autopilots support all 3 camera control options. For example (quite old) DJI A2 does support all three options, but newer (starting from Phantom 3 and up to M600) cameras support only triggering in waypoints and by time. DJI promised to implement triggering by distance, but it’s not available yet.

Here are some benefits and drawbacks for all three methods:

Table 1: Benefits and Drawback for camera triggering methods

In conclusion:

• Trigger in waypoints should be preferred when possible
• Trigger by time should be used only if no other method is possible
• Trigger by distance should be used when triggering in waypoints is not possible to use

To select triggering method in UgCS Photogrammetry tool accordingly, use one of three available icons:

• Set camera mode
• Set camera by time
• Set camera by distance

-Glibal control

Drones, e.g., DJI Phantom 3, Phantom 4, Inspire, M100 or M600 with integrated gimbal, have the option to control camera position as part of an automatic route plan.

It is advisable to set camera to nadir position in the first waypoint, and in horizontal position before landing to prevent lenses from potential damage.

To set camera position, select the waypoint preceding the photogrammetry area and click set camera attitude/zoom (Figure 12) and enter “90” in the “Tilt” field (Figure 13).

Figure 12: Setting camera attitude

Figure 13: Setting camera position

As described previously, this waypoint should be a Stop&Turn type, otherwise the drone could skip this action.

To set camera to horizontal position, select last waypoint of survey route and click set camera attitude/zoom and enter “0” in the “Tilt” field.

-Turn types

Most autopilots or multirotor drones support different turn types in waypoints. Most popular DJI drones have three turn-types:

• Stop and Turn: drone flies to the fixed point accurately, stays at that fixed point and then flies to next fixed point.
• Bank Turn: the drone would fly with constant speed from one point to another without stopping.
• Adaptive Bank Turn: It is almost the same performance like Bank Turn mode (Figure 13), but the real flight routine will be more accurately than Bank Turn.

It is advisable not to use Bank Turn for photogrammetry missions. Drone interprets Bank Turns as “recommendation destination waypoint”. The drone will fly towards this direction but will almost never pass through the waypoint. Because drone will not pass the waypoint, no action will be executed, meaning the camera will not be triggered, etc.

Adaptive Bank Turn should be used with caution because a drone can miss waypoints and, again, no camera triggering will be initiated.

Figure 14: Illustration of typical DJI drone trajectories for Bank Turn and Adaptive Bank Turn types

Sometimes, adaptive bank turn type has to be used in order to have shorter flight time compared to stop and turn. When using adaptive bank turns, it is recommended to use overshot (see below) for the photogrammetry area.

-Overshot

Initially overshot was implemented for fixed wing (airplane) drones in order to have enough space for manoeuvring a U-turn.

Overshot can be set in photogrammetry tool to add an extra segment to both ends of each survey line.

Figure 15: Adding 40m overshot to both ends of each survey line

In the example (Figure 15) can be seen that UgCS added 40m additional segments to both ends of each survey line (comparing to Figure 8).

Adding overshot is useful for copter-UAVs in two situations:

1. When Adaptive Bank Turns are used (or similar method for non-DJI drones), adding overshot will increase the chance that drone will precisely enter survey line and camera control action will be triggered. UgCS Team recommends to specify overshot that is approximately equal to distance between the parallel survey lines.
2. When Stop and Turn type is in use in combination with action to trigger camera in waypoints, there is a possibility that before making the shot, drone will start rotation to next waypoint – it can result in having photos with wrong orientation or blurred. To avoid that, shorter overshot has to be set, for example 5m. Don’t specify too short value (< 3m) because some drones could ignore waypoints, that are too close.

Figure 16: Example of blurred image taken by drone in rotation to next waypoint

-Takeoff point

It is important to check the takeoff area at site before flying any mission! To better explain best practice on how to set takeoff point, first discuss an example of how it should not be done. Supposing that the takeoff point in our example mission (Figure 17) would be from the point marked with the airplane-icon, and drone pilot would upload the route on the ground with set automatic mission for automatic take-off.

Figure 17: Take-off point example

Most drones in automatic takeoff mode would climb to low altitude about 3-10meters and then fly straight towards the first waypoint. Other drones would fly towards first waypoint straight from ground. Looking closely at the example map (Figure 17), some trees between the takeoff point and the first waypoint can be noticed. In this example, the drone more likely will not reach a safe altitude and will hit the trees.

Not only the surroundings can affect takeoff planning. Drone manufacturers can change drones elevation behavior in drone firmware, therefore after firmware updates it is recommended that you check drones automatic takeoff mode.

Also, a very important consideration is that most small UAVs use relative altitude for mission planing. Altitude counted relatively according to first waypoint is a second reason why an actual takeoff point should be near the first waypoint, and on the same terrain level.

UgCS Team recommends placing the first waypoint as close as possible to actual takeoff point and specifying a safe takeoff altitude (≈30m in most situations will be above any trees, see Figure 18). This is the only method that warrants safe takeoff for any mission. It also protects from any weird drone behaviour and unpredictable firmware updates, etc.

Figure 18: Route with safe take-off

-Entry point to the survey grid

In the previous example, (see Figure 18), it can be noticed, that after adding the takeoff point, the route’s survey grid entry point was changed. This is because if additional waypoint is added subsequently to the photogrammetry area, UgCS will plan to fly the survey grid starting from nearest corner to the previous waypoint.

To change the entry point to survey grid, set additional waypoint close to the desired starting corner (see Figure 19).

Figure 19: Changing survey grid entry point by adding additional waypoint

-Landing point

If no landing point will be added outside the photogrammetry area after the survey mission, the drone will fly and hover in the last waypoint. There are two options for landing:

1. Take manual control over the drone and fly to landing point manually,
2. Activate the Return Home command in UgCS or from Remote Controller (RC).

In situations when the radio link with the drone is lost, for example if the survey area is large or there are problems with the remote controller, depending on the drone and it’s settings, one of these actions can occur:

• Drone will return to home location automatically if lost radio link with ground station,
• Drone will fly to last waypoint of survey area and hover as long as battery capacity will enable that, then: drone will perform emergency landing, or it will try to fly to home location.

The recommendation is to add an explicit landing points to the route in order to avoid relying on unpredictable drone behavior or settings.

If the drone doesn’t support automatic landing, or the pilot prefers to land manually, place the route’s last waypoint over the planned landing point with an altitude for comfortable manual drone descending and landing above any obstacles in the surrounding area. In general 30m is best choice.

-Action execution

Photogrammetry tool has a magic parameter “Action Execution” with three possible values:

• Every point
• At start
• Forward passes

This parameter defines how and where camera actions specified for photogrammetry tool will be executed.

The most useful option for photogrammetry/survey missions is to set forward passes, the drone will make photos only on survey lines, but will not make excess photos on perpendicular lines.

-Complex survey areas

UgCS enables photogrammetry/survey mission planning for irregular areas, having functionality to combine any number of photogrammetry areas in one route, avoiding splitting the area in separate routes.

For example, if a mission has to be planned for two fields connected in a T-shape, and if these two fields are marked as one photogrammetry area, the whole route will not be optimal, regardless any direction of survey lines.

Figure 20: Complex survey area before optimisation

If the survey area is marked as two photogrammetry areas within one route, survey lines for each area can be optimised individually (see Figure 21).

Figure 21: Optimised survey flight passes for each part of a complex photogrammetry area

Step three: deploy ground control points

Ground control points are mandatory if the survey output map has to be precisely aligned to coordinates on Earth.

There are lots of discussions about the necessity of ground control points in cases when a drone is equipped with Real Time Kinematics (RTK) GPS receivers with centimeter-level accuracy. This is useful, but the drone coordinates are not in themselves sufficient because, for precise map aligning, image center coordinates are necessary.

Data processing softwares like Agisoft Photoscan, Dronedeplay, Pix4d, Icarus OneButton and others will produce very accurate maps using geotagged images, but the real precision of the map will not be known without ground control points.

Conclusion: ground control points have to be used to create survey-grade result. For a map with approximate precision, it is sufficient to rely just on RTK GPS and the capabilities of data processing software.

Step four: fly your mission

For carefully planned missions, flying it is the most straightforward step. Mission execution differs according to the type of UAV and equipment used, therefore it will not be described in detail in this article (please refer to equipment’s and UgCS documentation).

Important issues before flying:

• In most countries there are strict regulations for UAV usage. Always comply with the regulations! Usually these rules can be found on web-site of local aviation authority.
• In some countries special permission for any kind of aerial photo/video shooting is needed. Please check local regulations.
• In most cases missions are planned before arriving to flying location (e.g., in office, at home) using satellite imaginary from Google maps, Bing, etc. Before flying always check actual circumstances at the location. There could be a need to adjust take-off/landing points, for example, to avoid tall obstacles (e.g., trees, masts, power lines) in your survey area.

Step five: image geotagging

Image geotagging is optional if ground control points were used, but almost any data processing software will require less time to process geotagged images.

Some of the latest and professional drones with integrated cameras can geotag images automatically during flight. In other cases images can be geotagged in UgCS after flight.

Very important: UgCS uses the telemetry log from drone, that is received via radio channel, to extract the drone’s altitude for any given moment (when pictures were taken). To geotag pictures using UgCS, assure robust telemetry reception during flight.

For detailed information how to geotag images using UgCS refer to UgCS User Manual.

Step six: data processing

For data processing, use third party software or services available on the market.

From UgCS Team experience, the most powerful and flexible software is Agisoft Photoscan (http://www.agisoft.com/), but sometimes too much user input is required to get necessary results. The most uncomplicated solution for users is online service Dronedeploy (https://www.dronedeploy.com/). All other software packages and services will fit somewhere between these two in terms of complexity and power.

Step seven (optional): import created map to UgCS

Should the need arise for the mission to be repeated in the future, UgCS enables importing the GeoTiff file as a map layer and using it for mission planning. More detailed instructions can be found in UgCS User Manual. See the result of an imported map created using UgCS photogrammetry tool imported as GeoTiff file in Figure 22.

Figure 22: Imported GeoTiff map as layer. The map is output of a Photogrammetry survey mission panned with UgCS

Visit the UgCS homepage

Download this tutorial as a PDF

If you liked this tutorial, you may also enjoy these:

• Programming for robotics: Introduction to ROS
• Python programming your NAO robot
• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Adjust how you learn and quickly pick up robotics programming
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

This handy video-tutorial course gives an introduction to the Robot Operating System (ROS), including many of the available tools that are commonly used in robotics. With the help of different examples, the tutorials offer a great starting point to learn programming robots. You will learn how to create software including simulation, to interface sensors and actuators, and to integrate control algorithms.

The course consists of a guided tutorial and exercises with increasing level of difficulty working with an autonomous robot. We provide recordings of the lectures and give an introduction to the exercises. From the course website, you can download all the material including exercise sheets and templates, and use the provided Virtual Machine (VM) image to start programming right away.

Objectives:

• ROS architecture: Master nodes, topics, messages, services, parameters and actions
• Console commands: Navigating and analyzing the ROS system and the catkin workspace
• Creating ROS packages: Structure, launch-files, and best practices
• ROS C++ client library (roscpp): Creating your own ROS C++ programs
• Simulating with ROS: Gazebo simulator, robot models (URDF) and simulation environments (SDF)
• Working with visualizations (RViz) and user interface tools (rqt)
• Inside ROS: TF transformation system, time, bags

Course 1:

Course 2:

Course 3:

Course 4:

Download the full presentation here.

P. Fankhauser, D. Jud, M. Wermelinger, M. Hutter, “Programming for Robotics – Introduction to ROS”, ETH Zurich, 2017. DOI: 10.13140/RG.2.2.14140.44161

If you liked this tutorial, you may also enjoy these:

• Python programming your NAO robot
• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Sticky business: Five adhesives tested for 3D printing
• Adjust how you learn and quickly pick up robotics programming
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Tutorial 1: Speech

Helpful hints:

Follow these steps to create a box that we could put our python code into, and have our robot speak: “Hello”. “Hello I am a robot”.

1. Right click the main section box and go to “add new box”
2. Create a name for the box
3. Input main image
4. Select the type of box

Now we’re having a look at the script. Go into the scripts and click ok. And there we have our NAO little box. If we click into it, we have the python code. What we are going to do is change some of the codes to get our robot to speak.

1. Double click the box
2. Select the piece of code you need to alter to get our robot to speak(“def onInput_onStart(self)”)
3. Take out the pass section
4. Type in “ttsproxy = ALProxy(“ALTextToSpeech”)”. (Make sure you get the comment in there).
5. Underneath that, we are going to put “ttsproxy.say” and add the comment “Hello, I am a robot”.

The first line could create an object that gives us access to the robot’s text to speech capabilities. This object is assigned to the variable ttsProxy. We can access the object later through the name ttsProxy. I am going to mount the top line, and the alspeech is there. The second line calls a different function site. This function takes an argument “Hello, I am a robot” and then he uses a string or sequence of characters between two double quoted marks. The robot will speak the string that will pass to the say message aloud.  There are your simple first steps into python and choreograph.

You can now, hopefully, get your NAO robot to talk by using your python programming skills. Now have a play with it’s behavior to see if you can get your NAO to say a longer sentence, or maybe even sing a song. Have a play, and use your imagination!

Tutorial 2: Walking

Helpful hints:

No we’re going get your robot to walk using Python. The NAO can be program to walk to any point using the in-build Python Programming Language instead of using the drag and drop boxes in the choregraphe. Python is more expressive and allows you to do things like computing trigonometric calculations on the robot.

1. Right click the main section box and go to “add new box”
2. Create a name for the box
3. Input main image
4. Select the type of box

First, we want our robot to stand up. Drag a stand up icon into the box. I am actually using the NAO key so the 3D robot in the robot view, and this because we need to put the robot on connecting so we can see what’s going on quite clearly. Oversee when you are using this on the real robot and never forget to trigger on the motors and connect everything up properly.

Open the Code Box

1. Go to “def onInput_onStart(self):”
2. Write “motionProxy = ALProxy(“ALMotion”)”
3. On the next line write “walkTo(0.2,0.0,0.0)”
4. End it with “self.onStopped()”

The first line is a proxy called ALmotion, which what is we created here, which allows us to call the motion functions up. The second line is the walkTo, which moves the robot to a specific distance. Now, look on the decimal places: see that .2 is X and Y and the radius, so he’s going to move .2 forward. Then if I start playing with the other numbers here, he’ll move to the Y radius and then to the other turning point as well. The final bit is the self.on. To stop this, we will call and tell the choregraphe to stop the procedure when it is at the end and just simple loop and loop and loop.

If we are good, we should see this now on the 3D NAO. So if I hit play, we can see it walking along. Well done guys, you can now get your NAO Robot to walk by using
your Python programming skills. Tread carefully!

If you liked this article, you may also enjoy these:

• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Sticky business: Five adhesives tested for 3D printing
• Meet RoboGabby: She’s on a mission to teach more girls about robotics
• Adjust how you learn and quickly pick up robotics programming
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The Arm

The design I used is based on industrial pallet-packing robots, and at its core has three degrees of freedom, or ‘axes’ on which it can move. Think left to right, in and out and up and down; it basically means that the arm can move in three different ways. The GIFs below show the different types of movement: the entire arm can swivel on its base, the “main arm” translates the elbow in and out from the centre, and the “actuator” drives the forearm, which in turn translates the end-effector up and down. A 3-DOF arm like this has the handy property of having a unique solution for each possible position of its end-effector, which made the IK task approachable to someone like me, who’s never done this type of thing before.

Swing

Main

Actuator

Like many well-studied problems, IK has generic solutions, but these are numerically-based and computationally complex. I wanted a simple and efficient solution, so I decided to approach the control of my arm analytically, using geometric principals and logic. The result of this work was a graphical Python application, which connected to the arm via serial and allowed the user to move the goal position of the end-effector by clicking and dragging, or by sending commands from another application running on the same computer (this IK application made up a key part of my project, but that’s a topic for another time).

Here’s a quick look at what I ended up with:

Heads up: The code in this post is written in Python with Numpy, and there’s maths. I won’t pretend that any of the ideas presented here are optimal (or good, for that matter), but hopefully you’ll be able to scrounge something useful!

Let’s go already

To start with, let’s consider the “swing” action of the arm, as it rotates on its base. Since the main arm, elbow and forearm are all constrained to a single plane (look at the GIFs above: only the swing rotation affects the left-right position of the wrist), all we really have to do here is line up that plane – a line from our top-down perspective – with the goal point, ignoring the goal’s vertical position. If this sounds easy, you’re right: it’s plain old trigonometry. Unfortunately for me, a quirk of my design left the arm’s operation plane just off-centre of the axis of rotation, which made the solution much less obvious.

Using the naive trigonometric approach here results in pointing off to the left of the actual goal (right if the shoulder sits on the other side), and the gap is big enough that this is a real problem. I took a shot at solving this geometrically, but quickly decided it wasn’t worth my time figuring it out. In the end, I resorted to an extremely simple numerical technique called a binary search. I’ve used this in the past for other tough geometric problems, like finding an intersection between a Bezier curve and a circle.

In a binary search (also known as the “Bisection Method”), you essentially come up with arbitrary upper and lower bounds for your variable – in this case, theta – and then examine the midpoint to find whether it’s larger or smaller than your target. If it’s larger, then the midpoint becomes your new upper bound; if it’s smaller, the midpoint is your new lower bound. Repeat until you’ve bounded your variable with sufficient accuracy. A binary search is conceptually simple and computationally cheap, and converges fairly quickly: you halve your search space each iteration. I found 3-5 iterations got me within 0.3 degrees of the true value, well within the limits of what my servo motors could practically target.

On to the solution! What we’re actually searching for is the point at which the vector representing our arm is lined up with the vector from the arm’s shoulder joint to the goal: where the angle between these two vectors is zero. Normally, when taking the angle between two vectors (call them A and B) you get the absolute angle out, with no directionality. However, with this search method it’s important to have a signed distinction between “too high” and “too low”. To get around this, I wrote a simple function to find a signed angle between two vectors, with negative showing that A points counter-clockwise to B, and positive showing that A points clockwise from B:

This works by taking a perpendicular vector from B (pointing clockwise), and dot-ing it with A to see how much they line up (using the dot product for
vector projection).

If A points clockwise of B then np.dot(A, perp) will be positive, and vice-versa.

Note that the normalize() function simply sets a vector’s length to 1, while keeping it pointed in the same direction. Mathematically, it’s just doing: A=|A|^-1 (np.clip() just keeps the dot product between 1 and -1).

With that, we can now perform our binary search and find a decent approximation to the target angle. We need a reasonable starting point, so let’s just use the angle from the origin to the goal (the naive approach I mentioned above). Numpy provides a very useful function for finding the angle np.arctan2(y, x) which resolves the arc-tangent of in the correct quadrant.

Now, it’s important to check which side the shoulder sits on: this lets us decide whether to use start_theta as an upper or lower bound. This is as simple as testing the signed angle between the shoulder offset vector (from the origin) and a vector pointing straight forwards. At the same time, we can also establish the other bound by taking the absolute value of this angle and adding or subtracting it from start_theta, since this angle to the shoulder is always going to be greater than the error of start_theta.

With our initial bounds established, the actual search loop is pretty straightforward. Note the use of an iteration counter: it’s only there in case something goes wrong (e.g. if we input an unsolvable configuration), since in normal operation the loop exits once the desired accuracy is reached. We iteratively update self.swing, which will hold the new target swing angle at the end of the search.

Note that the self.shoulder(angle) function just generates the top-down (x, z) position of the shoulder joint for a given swing angle. Besides the swing angle, there’s one other important result from this stage of IK: the radial distance, from the shoulder joint to the end-effector. The calculation is simple enough:

goal_radial = np.linalg.norm(self.goaltd – self.shoulder(self.swing))

Round in circles

We know which way to point: now, let’s consider the movement of the arm in its ‘plane of operation’, which covers the goal end-effector position and the line extending vertically (+y) from the shoulder joint. The main arm and forearm both move across this plane. The trick here is that we’ll still be using a 2D coordinate system, basically a local frame of reference, where our y value remains the same but the x value is the radial distance of the point we’re interested in. For the goal, we computed this radial distance above in the top-down IK, so we can go ahead and construct goalpl – the ‘planar’ goal – which lies at: goalpl=(goal_radial/goal_y).

Side-on, the main arm and forearm form an upright triangle between the shoulder joint and goalpl with the elbow joint sitting at the peak. It’s important to note that the arm’s sections are different lengths, with the forearm being a bit longer. Now, in this plane we’re solving given the end effector’s position, and the origin is static. That just leaves the position of the elbow joint to solve for, which we can do geometrically, since it’s at a fixed distance from both the origin and effector. If we draw two circles, one fixed about the origin with radius = main arm length, and one centred on the end-effector with radius = forearm length, then the elbow must be at one of the two possible intersections between these circles:

Let’s look at a simple Python class which deals with this intersection calculation. There’s a few possible cases for circle-circle intersection, most of which invalidate our solver: if the circles do not contact, either by being separated or in the case that one contains another, then there is no valid solution; if the circles are coincident (impossible for this design, since the radii differ) then there are infinite possible solutions; however, when the circle borders intersect, we can find two distinct intersection points.

For any case we’re interested in, it’s the top intersection that matters. This means we can just pick the one with the largest y value, and call it a day:

And – that’s it! We have the swing angle, and a valid position for the elbow joint to match the shoulder and goal end-effector positions. That’s all we need to define a valid pose for the arm. If you need to, you can reconstruct the positions in 3D space:

Wrapping up

If you’ve made it this far, I hope you’ve found this interesting and/or enlightening. Want to see how it all fits together? I’ll be uploading the full IK application to GitHub here, along with all the design files for the robotic arm. If you’re feeling ambitious, you should be able to 3D-print and build your own to play around with

Other notes:

• I didn’t design the arm completely from scratch: check out the original “LiteArm” design on Thingiverse.
• The IK application uses PyGame, a no-frills SDL wrapper for Python, to open and draw to the graphical window. I’ll probably switch to a meatier library for further development, but this did the job of basic “drawn-to-screen” stuff nicely.

Written by Alistair Wick for the University of Bristol Engineering Maths blog. The original post, and future follow-ups, are available on Alistair’s website.

If you liked this article, you may also enjoy these:

• TrotBot tackles rough terrain
• Sticky business: Five adhesives tested for 3D printing
• Meet RoboGabby: She’s on a mission to teach more girls about robotics
• Adjust how you learn and quickly pick up robotics programming
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Hi, I’m Ben. I was a member of the team that developed a new walking mechanism, TrotBot, that we eventually scaled up to the size of a mini-van. We were inspired to begin this project in 2012 when we discovered Theo Jansen’s Strandbeest. This led us to embark on a LEGO building spree, attempting various existing walking mechanisms.  However, they performed poorly on rough terrain, so we decided to design our own. We wanted our design to have lifelike motion and to be functional on rough terrain. After a couple months we had designs and LEGO iterations of TrotBot, a walking mechanism with the gait of a galloping horse, and a very high footstep to tackle obstacles. Below is a simulation of the linkage we discovered compared to Strandbeest and Klann’s Mechanical Spider.

After tinkering with our new linkage in LEGO, it wasn’t long before we, somewhat naively, decided to scale it up to the size of a minivan. This turned out to be a more difficult challenge than we originally anticipated, like asking a bridge company to build a bridge, but with moving supports that occasionally lift off the ground. Despite a rat’s-nest of problems to solve, we finally managed to get this machine walking after two summers of hard work. Below is a clip of TrotBot 1.0.

I scaled back down to LEGO scale, using the insights gained from the scale-up, and the functionality of my builds improved enough that I decided to post the mechanism and build plans. For an example of how TrotBot’s high foot-path improved its functionality, below you can see a LEGO TrotBot tackling an obstacle course.

I’ve posted detailed plans on how I built TrotBot in LEGO, some of the engineering insights behind the design, as well as some of what we learned throughout the process of creating TrotBot at DIY Walkers, where I will continue to post my progress with future builds.  If you have insights into TrotBot or walker projects you’d like to share, or if you are a teacher hoping to motivate your students to take on a hard STEM challenge, please visit the site and get in touch!

If you enjoyed this article, you may also want to read:

• Work, play and STEM: How robotics will bridge the gap
• Meet RoboGabby: She’s on a mission to teach more girls about robotics
• Motivational posters with a clever robotics twist
• Teaching the teachers: Study focuses on how teachers perceive educational robots in classrooms
• Adjust how you learn and quickly pick up robotics programming

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

We compared five adhesives for 3D printing applications on a Wanhao Duplicator i3. We’ll print a PLA (polylactide) cube, 1х1х1 cm in size.

1.  NELLY LACQUER: We spread it on the clean surface of the table and turn on the Print Mode. The printed model is very well stuck—a pallet-knife is needed to separate it from the surface of the table. The adhesion result is very good.

2.  THE 3D GLUE: We apply it to the table with a cloth, wait for the table to warm up, then start printing. The result is similar to Nelly lacquer. We can’t separate the printed model from the table without any tools. The bottom of the cube is smooth.

3.  PVA GLUE: We spread a small quantity on the table and let it dry before printing. It is not easy to tear the model from the table. The bottom surface of the cube is rather rough and has PVA glue stains.

4.  GLUE STICK: We try to apply a thin layer of it to the hot platform but, because the adhesive is very thick, we can’t do it using a microfiber cloth. To separate the cube from the platform we use a pallet-knife. The bottom surface of the model and the platform surface have white glue stains. Now we need to clean the table and apply an adhesive anew.

5.  STICKY TAPE (an analogue of a blue Scotch): Finally, we paste one layer of it to the table, carefully smoothing it out to avoid blistering. This adhesive is no good for big ABS models as they tend to break away from the table along with the tape.

We try to accurately separate the cube from the table to prevent the tape from peeling off, but part of the tape stuck too well to the cube and came off the table along with the model. Now we’ll have a naked spot (without the tape layer) on the table when we print next time.

Conclusions

• The lacquer and 3D glue give the best results; the bottom surface of the printed models is clean and smooth, without any traces of adhesives.
• The glue stick stains the models.
• The Scotch tape tends to stick to the model and comes off from the table partially or fully.
• For ABS printing we recommend using a closed 3D printer and a strong adhesive, like NELLY.

If you liked this article, you may also want to read these other articles on 3D printing:

• Foundry tool: Multi-material designing for 3-D printing
• 3-D printing hydraulically-powered robots, no assembly required
• AI, deep learning and 3D printing produce ‘The Next Rembrandt’
• Rising Media acquires RoboUniverse and Inside 3D Printing shows

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Source: RoboGabby/YouTube

Gabby has five goals to achieve this school year. The first goal sounds like any other young middle schooler: get straight As in her classes. Goal number two: get accepted into her school’s robotics team, followed closely by number three: to own her first VEX IQ Robotics kit. That’s when you may realize, Gabby is an extraordinary middle schooler!

Gabby, or “RoboGabby,” is a budding science communicator and roboticist, aspiring to be a role model and STEM ambassador. She began creating tutorial videos to encourage girls to give robotics a try, born from her own experience witnessing the lack of girls involved with STEM at her school, and specifically after an all-girls robotics camp was cancelled due to lack of sign-ups.

“My dad found other robotics camps for me to attend, but there were only about 4 girls who participated, in total, of all the camps I attended — including myself,” she said. “I constantly asked my parents ‘why aren’t more girls involved in STEM? My dad suggested that I try to inspire other girls to get involved. I was very nervous when I first started and scared that people would not like my videos. But everyone online and at my school seems to love my videos,” she said.

Gabby was introduced to programming and robotics by her dad at the age of 8 and enjoyed being able to “explore what a robot can do through code.” She loves working with her hands, having now been to several computer and technology camps, as well as, three robotics camps.

“I started out learning the Python programming language, and I did not quite understand why he was making me learn it.” she said. “Eventually, I started doing fun things with code and began to like it. From 8 to 10 years old, I learned how to program in Python, a little bit of Java, and learned block programming with Scratch. When I was 10 years old, I purchased my first robotic arm using my own Christmas money.”

“I now consider myself a role model, because I am getting recognition from girls and boys all the time,” she said. “I am still making tutorial videos with Robot Virtual Worlds, by Robomatter, and I’m working on making tutorial videos for another robot simulation software called CoderZ. I also plan on making Java programming tutorial videos in the future.”

“I still do not have an actual physical robot, but I’m hoping to get a Vex Programming or Dual Control Starter Kit some day and start making tutorial videos with that, as well. I’ve discovered that being different was something cool, rather than something to be embarrassed about,” she said.

From all of us here at Robohub, we look forward to seeing more of your creative tutorials Gabby! Keep creating!

You can follow RoboGabby’s tutorials on her YouTube channel, or on any of her social media channels, like Twitter or Facebook or Instagram.

If you enjoyed this article, you may also want to read:

• Work, play and STEM: How robotics will bridge the gap
• 25 women in robotics you need to know about – 2016
• Motivational posters with a clever robotics twist
• Teaching the teachers: Study focuses on how teachers perceive educational robots in classrooms
• Adjust how you learn and quickly pick up robotics programming

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Credit: Ricardo Tellez

“The biggest obstacle preventing robotics going mainstream is not having good programmers able to program robots,” Brian Gerkey, CEO of the Open Source Robotics Foundation.

Learning to program robots is difficult.

I mean, real robots.

I am not talking about learning to program Lego and the like. I’m talking about making a Reem robot move around a shopping mall helping people to carry their bags. Or a Fetch robot cooperate on a warehouse with humans helping them to fetch the packages.

Yes, programming of these robots is difficult.

In order to simplify the programming of those robots, frameworks have been created:

1. ROS
2. OROCOS
3. YARP
4. URBI
5. etc…

Those are the real thing. They are frameworks that are installed in real robots and used to make them work. I’m not talking about using Matlab for programming your algorithm in a robot. Matlab is not used in real life, only in academia for non-replicable results. Here we are talking about the real thing.

But don’t be get me wrong. Even if those frameworks are quite complex, anybody can learn to program robots. The question is: how long does it take?

Why does it take so long to learn how to program robots?

We have identified a series of problems that prevent quick learning of robotics.

You don’t have access to robots to play with

There are many different robots available and they are really expensive. Acquiring, maintaining, and explaining the hardware are real problems that limit class size and the learning experience of the students.

As Nikolaus Correll, Rowan Wing and David Coleman indicate in their paper, A One-Year Introductory Robotics Curriculum for Computer Science Upperclassmen:

Only a subset of the course material of a robotics curriculum can be assessed conceptually or mathematically. Demonstrating a thorough understanding of an algorithm requires implementation. Unlike purely computational algorithms, which can be assessed automatically using test-input, authentic assessment scenarios are difficult to create for a robotics class.

Hence, students can hardly practice what they are learning.

We strongly believe in the CONSTRUCTIONISM approach. Defined by Seymor Papert around 1980, the constructionism approach advocates for learning by making. That is, you are not learning when somebody is teaching you, but you are learning when you practice.

If you don’t have a robot to practise with, you cannot learn properly and at full speed.

Concepts are complex to understand

The concepts of learning how to make a robot work are complex (how to make a robot grasp, how to build a navigation program, how to make it reason, etc.), which makes the learning curve very steep.

How many of you out there use ROS? Ok, how would you explain ROS to others who know nothing about it? Difficult, isn’t it?

The most meaningful unit in robotics for a newcomer is making the robot do something. Given the student’s program to make the robot do something, the robot either does it or not. This is very clear to understand. You push this button and the robot does that (or not, if you have a failure in your program). The feedback of your performance is very clear and straight.

It is not the same to get the feedback about the robot doing an action than to do it because the compilation said you forgot to include something in the CMakeLists.

The longer it takes to associate the errors with the result in the robot, the more difficult it is for the student to close the loop of understanding, and hence, learn why or why not this works that way. This is the concept of delayed reward. The longer delayed reward, the more difficult to understand the relation cause-effect and the longer it will take to learn the concept associated.

The challenge is then to provide such feedback very quickly, even if there are many steps and concepts involved in a robot action. Hence, teaching materials cannot start by indicating how to install everything, how to compile, or how to create a download new packages.

Nobody needs to know the definition of ROS to start using it. It is not a good idea to start teaching ROS by defining what a package is, or what a workspace is, then how catkin works, then what is the ROS_MASTER_URI, etc.

All of those are important, but not at this early stage. Going at that pace will take ages to make the robot do something, and for the student to understand.

Documentation made by teachers for teachers

We found that documentation, tutorials and classes are made by people who already know robotics for people who already understand robotics. I call these type of tutorials reference tutorials. Those tutorials basically start by describing concepts one-by-one, following a hierarchy of well-defined definitions by people who already understand the subject. If you ask the reason of that concept or for what do you need it, they will tell you that you will understand later (but later is too late!).

Generally speaking, robotics is sometimes taught as if it were for the instructor. It is very structured with a clear concepts hierarchy. It is a problem that documentations of robot programming frameworks work as a reference tutorial for the engineers that wrote it. Of course, you can learn from those but it is a lot harder and much slower (I learnt ROS that way).

I do believe we do not work this way. One thing is the way we structure information once we know the whole space, and another is the way we structure it when we are learning. Very different things.

The solution we propose

The method we propose is to make the students forget as many concepts as possible and concentrate on doing things with the robots. Complex concepts will be taught later when the student has the base. This approach is based on how we learn and not in how we order our thoughts once we have them.

We propose a solution to speed the learning that provides the following steps:

1. In media res. That is the trick Star Wars used to make the movie more appealing (new Star Trek movies have copied successfully, I think).

This means, start in the middle, not the edges. The idea is to put the student in the middle of the knowledge to learn and make it move to the sides, discarding a lot of things in the way to the side.

2. Learning by doing. Not by reading or by typing, by working on a robot.

This is the successful case of Code Academy. There you can learn to code by doing and quickly observing the result of what you have done.

We must have the same type of environment for robotics learning. Do, then observe on the robot the result of your program.

3. A step by step guide. Showing only what is required for the next action of the robot, and removing out of the equation what is not required/useful at that specific learning moment (even if important for the success of the step).

For example, you must have a catkin_ws in order to launch a ROS package, but we do not believe that we must explain to a new student how to create a catkin_ws. Instead, we provide it done ready to be used, and once he becomes used to it, then we show how to create your own.

Implementation

At The Construct, we have integrated all those ideas into a platform that allows students to quickly learn (in 5 days) any concept of robotics programming. I mean, for real robots!

Our platform is called Robot Ignite Academy, and we like to call it the Code Academy for Robotics. We integrated step by step explanations with a robot simulation and program development environment, everything inside a web browser (hence you can learn using any type of device).

At Ignite Academy, students will learn complex robotics concepts by:

1. Using simulations of the real robots. Many. Examples include Turtlebot-2, Husky, Sphero, Wam, quadcopters… even BB-8! Students use a simulation of the robots as if they had the robot.
2. Use a step-by-step lesson process that engages the student on doing something with the robot after every step, in order to check that he is understanding the explanation.
3. Use the description not only to explain, but also to do. Include interactions in the explanation. For example, generate graphics dynamically on the explanation based on what the robot is doing at that moment.
4. Forget about certain basic concepts (for example, what is ROS… who cares? You don’t need to know what is ROS in order to use it and make a robot work) We believe that once you have mastered the real thing, getting those concepts will be easy peace. But if you put them on the pile of things you have to learn together with the others, it makes the whole thing difficult to swallow.

All that without having to install anything in your computer, and with simulations at the core of the learning system, not only as something additional.

In our system, the simulation is at the center. We provide different simulated robots that the student has to practice with. They use simulation as if they had the real robot, with the difference that they don’t have to care about building it. It is just provided and ready to be used.

Conclusion

We are providing the Ignite Academy platform online, for anyone. Students can use it for their personal learning. Teachers can use it for teaching during the class. Companies can use it to upgrade their workers.

What is also important to state is that you don’t have to care about installing anything. Connect using a web browser from any type of computer and start learning.

Source: Alberto Molina

Dtto v2.0 is a modular self-reconfigurable robot focused on bio-inspired locomotion mechanisms. This tutorial shows how to assemble a fully working Dtto Modular Robot module.

Please review Parts 1 and 2 before getting started.

This document is an illustrated guide on how to assemble a module (v2) for the Dtto Modular Robot: https://hackaday.io/project/9976-dtto-v20-modular-robot

Now, let’s jump in for Part 3!

Step 15

We need:

• 1 x LM317 Voltage Regulator
• 1 x 1200 Ohm Resistor
• 1 x 330 Ohm Resistor

Now we will configure our voltage regulator. Since we basically need the regulator for powering the servomotors, and since they can operate between 5 and 6 volts, set the regulator to as close to 6 volts as you can. You can use any other voltage regulator you want, considering the minimum current needs.

The LM317 Voltage regulator allows us to adjust the output voltage using some resistors. You can find the details and the math in its datasheet, widely available. Refer to the datasheet to be sure of the connections. The resistors that we choose were 330 and 1200 Ohms, resulting 5.8 volts. We can also use 100 and 390 Ohms, resulting in 6.1 volts. Check the schematics.

We need to power 5 servomotors with the output voltage from the regulator, so we need a small board to solder up the regulator, the resistors, and the cables. The dimensions of the board shouldn’t exceed 1*2.2cm, and it should fit in the module, as in the next pictures.

Check the schematics of the module, you can use the board to connect the cables that come from the batteries (switch), and the ones that connect to the Arduino.

Step 16

We need:

• 1 x Mini Switch

Now we are going to put a little switch to power on/off the entire module. Check the schematics.

For now, we’ll leave the switch hanging around and we will return to it later.

Step 17

We need:

• 1 x Arduino Nano v3.0

The Arduino Nano is the brain of our module, our robot. It is very important to test the Arduino before proceeding to the following steps.

In this step, we will test the Arduino Board and the connection with the computer and will program an example program: blink. Once tested, note that, in order to save some valuable space, we will need a pins-free board.

Now put the Arduino Nano into the module.

Step 18

We need:

• 1 x HC-05 Bluetooth module (step 11)

Now we are going to solder all the cables directly to the Arduino. Follow the schematics and try to keep the cables as short as possible. This is a tricky part, so you have to think how to put the cables to reduce length, allow all the components to fit, and allow the coupling mechanisms to work.

Check the schematics.

As you can see in the pictures, the Bluetooth module is “floating”. Since it’s on top of the Arduino, and we have to ensure that we can access the Arduino, cut the cables long enough. We’ve also used auxiliary boards to solder some cables together (grounds, etc.), for better cable organization.

Step 19

We need:

• 1 x WS2812 RGB LED
• 1 x Part 007-1 (step 10)

In this step, we will be mounting the fancy RGB LED of the robot. Cut the LED support to adapt it to the mounted LED. The cables should go, as in the picture (Note: the servo in this picture has no cables, we ran out of working ones).

Now it’s time to fix the printed part to the module body, using 2 screws. As you can see in the picture, the mini switch is just under the servo we just assembled. Stick the mini switch in a position that, when pressed, it doesn’t protrude from the module.

Step 20

We need:

• 1 x Part 003-1
• 1 x Part 006-1
• 1 x Part 008-1
• 1 x Part 010-1
• 2 x Li-Po Batteries

Just as we did in step 6, assemble part 003-1 to the other half of the module. Make sure that it rotates nice and easy.

Now that the module is almost assembled, think about where to put the batteries. The module is designed to host the batteries in the male part, at the back of the Arduino, but since we are not using the empty space in the female part, we are going to put the batteries in the female half. This way we get a better weight distribution.

First, put the cables through that will power up the system:

Then, mount Parts 006-1 and 008-1, just as we did in the other half. Once we have done this, we can proceed to insert the batteries. Stick them in like in the pictures:

We can now mount the Part 010-1.

Step 21

We need:

• 2 x Part 001-1
• 2 x Part 002-1

This is the final step and we are going to assemble the last parts.

First, test if they fit correctly. If not, sand down until you get them to fit nicely.

And this is the final result!

Now just program the Arduino with the available code and enjoy!

If you have any questions don’t hesitate to contact me, otrebla333@gmail.com.

Good luck with your modular projects!

This project and all of its files, images and texts are entirely licensed under the CC BY-SA 4.0 license. (http://creativecommons.org/licenses/by-sa/4.0/legalcode).

If you liked this article, you may also want to read:

• Make your own Dtto Modular Robot – Assembly instructions (Part 1 of 3)
• Make your own Dtto Modular Robot – Assembly instructions (Part 2 of 3)
• Hello Pepper: Getting started to program robots on Android
• NASA Robonaut 2 simulation: Placing an ISS panel in The Construct
• How to make a printed circuit board using a diode laser with a 3D printer

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Dtto v.2 module robot, DIY project, photo by Alberto Molina.

Dtto v2.0 is a modular self-reconfigurable robot focused on bio-inspired locomotion mechanisms. This tutorial shows how to assemble a fully working Dtto Modular Robot module.

Please review Part 1 before getting started.

This document is an illustrated guide on how to assemble a module (v2) for the Dtto Modular Robot: https://hackaday.io/project/9976-dtto-v20-modular-robot

Now, let’s jump in for Part 2!

Step 7

We need:

• 1 x Part 005-1
• 1 x Part 007-1
• 1 x Part 009-1
• 12 x Neodymium Magnets

In this step, we put the magnets on the male parts. It is very important to put all the magnets in the male part with the north pole facing the inside of the module. Put the magnets in the holes, then put a drop of glue, sticking it to the part. For this procedure, use a hot glue gun.

Repeat the procedure for the 3 parts:

Step 8

We need:

• 1 x Part 006-1
• 1 x Part 008-1
• 1 x Part 010-1
• 12 x Neodymium Magnets

As we did in the previous step, we will also put the magnets in the female parts. Remember to put all magnets with the north pole facing the outside of the module. If we do this correctly the male parts won’t stick between them, but a male and a female part will be magnetically attracted.

Step 9

We need:

• 3 x SG90 Servomotors
• 3 x Part 013-1
• 3 x Part 014-1
• 3 x Small rubber bands (dental braces)

In this step, we will prepare the coupling mechanism that holds the modules of the robot together.

First, insert the servo arm (previously cut, as in the pic) through the two printed parts. Ensure that these parts rotate freely around the servo arm. Put the servo in the 90° position (servo.write(90)).

Then, screw the parts, like in the picture, using the screw that came with the servo.

Finally, put the rubber band between the two parts.

Repeat the procedure with the other 3 SG90 servomotors.

Step 10

We need:

• 1 x Part 005-1 (step 7)
• 1 x Part 007-1 (step 7)
• 1 x Part 009-1 (step 7)

In this step, we will fix the servos that form the coupling mechanism to the printed parts.

First, insert the servo in the correct position.

Then, using the 2 screws that came with the servo, fix it to the part.

Repeat the procedure for the 2 parts left.

Step 11

We need:

• 1 x HC-05 Bluetooth module
• 1 x NRF24L01 radio module

In this step, we will prepare the Bluetooth and the radio module.

Remove the pin headers to solder wire directly, thus reducing the total volume of the modules. Be sure to use a flat cable from an old PC, because there won’t be much space for cables.

Step 12

We need:

• 1 x NRF24L02 radio module (step 11)

Now, we are going to stick the radio module to its final position.

Due to the lack of space and the need to reduce weight, we decided to simply stick the module with some hot glue. Be sure the module doesn’t make contact with the moving part in the center. The cables go to the other side of the module.

Step 13

We need:

• 1 x Part 009-1 (step 10)

Using 3 screws, mount the motor-equipped part 009-1 to the body of the module. Ensure that the servo cables don’t contact the coupling mechanism.

Step 14

We need:

• 1 x Part 005-1 (step 10)

In this step, we will mount the “base” of the robot, part 005-1, with the previously mounted motor.

It’s important to note the size of the cables connected to the radio module (step 11), because they have to fit between the “base” servomotor and the top part.

We need 4 screws for this step.

Tomorrow, we will conclude the tutorial with Part 3: configuring the voltage regulator, the Arduino Nano, and final part assembly!

This project and all of its files, images and texts are entirely licensed under the CC BY-SA 4.0 license. (http://creativecommons.org/licenses/by-sa/4.0/legalcode).

If you liked this article, you may also want to read:

• Make your own Dtto Modular Robot – Assembly instructions (Part 1 of 3)
• Hello Pepper: Getting started to program robots on Android
• NASA Robonaut 2 simulation: Placing an ISS panel in The Construct
• How to make a printed circuit board using a diode laser with a 3D printer

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Source: Alberto Molina

This tutorial shows how to assemble a fully working Dtto Modular Robot module.

This robot is inspired by the MTRAN III, by the National Institute of Advanced Industrial Science and Technology (AIST), Japan.

We strongly recommend you have a rasp (also called: nail file, emery board, or sandpaper) close by and a set of precision blades.

First of all, we need to print all the robot parts. For this type of project, we think that ABS plastic is better as it is more machinable than others. All .stl files are available in the links below. You also have a link to a PDF file detailing how many parts we need, part numbers and some details (list of 3D printed parts). We will be using the part number to identify the parts. We also need the schematics file to check the connections.

Once we have all our parts printed, all supports removed and carefully sanded, we can proceed to the assembly of the module.

It has been fully printed with open-source 3D printers, uses open-source hardware, and has been designed entirely on FreeCad.

Note: In this tutorial, we won’t be assembling the IR detection system. This feature will be available as soon as possible.

We do not take any responsibility and we are not liable for anything occurring while assembling and playing with this modular robot.

All images copyright Alberto Molina.

Source: Alberto Molina

Step 1

This document is an illustrated guide on how to assemble a module (v2) for the Dtto Modular Robot: https://hackaday.io/project/9976-dtto-v20-modular-robot

In the project files, you will find the following documents:

• Bill of materials
• List of 3D printed parts
• Schematics
• STL files for printing

You will find all the documents also on:
https://github.com/otrebla333/Dtto-Modular-Robot

Components:

Step 2

We need:

• 2 x Part 004-1
• 1 x Part 011-1

First, we have to ensure these 3 parts fit perfectly. It is very important that the 004-1 parts rotate freely around the center. The easier they rotate the less energy is lost in friction, and the robot moves around better.

Source: Alberto Molina

Source: Alberto Molina

Step 3

We need:

• 2 x MG92B Servomotors
• 4 x 520 Ohms Resistors

The second thing to do is to adapt the MG92B servomotors to turn the full 180 degrees.

The method we used was to add a couple resistors to each servo control circuit:

• Take apart the back cover of the servomotor (4 screws).
• Carefully separate the control circuit from the inside. The potentiometer is located under the circuit (it may be covered by a black film). Once you can access the 3 pins of the potentiometer, you have to unsolder the first and the third cable (not the center one). Then, you have to solder a 520 ohms resistor to each of these two pins, and then solder the cable to the resistor. You can find tutorials on how to modify servomotors (search: modify servo 180 degree), but it’s important you use the 520 ohms resistors (in the pictures we used one 680 and one 2200 ohms resistor in parallel to get the 520 ohms).
• The control circuit does not fit the casing anymore, so we are going to stick it to the side of the metal case (see pictures). Protect the circuit from short-circuits with some tape on both sides.
• Repeat the procedure for the other servomotor.

Source: Alberto Molina

Source: Alberto Molina

Source: Alberto Molina

Step 4

We need:

• 2 x Part 004-1
• 2 x MG92B Servomotors (step 3)

In this step, we are going to fix the motors to the printed parts. The motor should fit correctly. We can use the screws that come with the motor, but we will have to cut them to the right length.

Source: Alberto Molina

Step 5

We need:

• 1 x Part 011-1
• 1 x Part 012-1
• 2 x Servo Arms (Cross)

Now we are going to fix the servo arms to the 3D printed part. This step is very important because we have to ensure that the servomotor is centered. If it is not centered, once assembled, the module motor won’t be able to rotate all the way.

The next image illustrates the result. When the servo is set to 90 degrees (servo.write(90)), the relative position of the parts should be like in the center picture. If the position is slightly deviated, like in the picture on the right, we will have to center the part.

Source: Alberto Molina

If we set the servo to 90 degrees and get a deviated position like in the picture on the right, we will have to un-center the servo arm, like in the following picture:

Source: Alberto Molina

Source: Alberto Molina

This is a critical part, so be sure to get a well-centered configuration. Once we get a well-centered part, we have to be completely sure that we have a correct configuration. We can connect the two parts and test the rotation with the “sweep” example in arduino IDE.

Source: Alberto Molina

Source: Alberto Molina

Once we finish with one servomotor, we can proceed to fix the other one.

Source: Alberto Molina

Source: Alberto Molina

At this moment, the module does not have a male or female part assigned. We will choose one half and we will name it male. The other will be the female half. Then, we have to put the cables of the female motor through the hole in the central part, to the male part. You have to ensure that the cables allow a free rotation of the parts.

Source: Alberto Molina

Step 6

We need:

• 1 x Part 003-1

Now we will start to assemble the male half of the module. First of all we have to ensure that the hole in the part 003-1 fits with the previously assembled part 012-1. Once sanded, we can attach the part 003-1 to the part 004-1 using 3 screws.

Source: Alberto Molina

Tomorrow will feature Part 2, putting magnets on the parts and servos!

This project and all of its files, images and texts are entirely licensed under the CC BY-SA 4.0 license. (http://creativecommons.org/licenses/by-sa/4.0/legalcode).

The Future of Social Robotics, by Nicolas Rigaud. Source: Margaret Maynard-Reid/Medium

I didn’t get to meet Pepper the humanoid Robot at Google I/O but I watched the video afterwards: A new development frontier: Android + Pepper the interactive robot. Love the robots’ dance! I was super excited to hear that Pepper will become available in the U.S. later this year, and Android developers can now program robots!

A few weeks after I/O, I attended a Seattle Java User Group meetup. That evening I met Pepper in person, and I learned about the Future of Social Robotics from Nicolas Rigaud (slides here).

Equipped with microphones, cameras, sensors and an intelligence to perceive emotions, Pepper was designed for social interactions. He has an Android tablet on his chest which is very convenient for displaying image, video and texts. Pepper is 4 feet tall (1.2 meters) and can connect to the network with either Wi-Fi or Ethernet. Pepper is multi-lingual in English, French, Japanese and Chinese.

Source: Margaret Maynard-Reid/Medium

I wanted to learn more about Pepper. So I followed the instructions from the Pepper SDK for Android and was able to quickly get set up and create my first robot Application with Android Studio.

Since I already have Android Studio installed, all I had to do was to install the Pepper SDK plugin before creating my first “Hello World” robot application. You can follow the official Getting Started guide for detailed instructions, and here I’m sharing with you a brief summary of the steps I took (note I already had JDK and Android Studio installed):

1. In Android Studio, install the Pepper SDK plugin; then the Robot SDK Manager icon appears on the Android Studio main tool bar
2. Click the Robot SDK Manager icon to get the Robot SDK and tools
3. Creating my first robot application

• First create a regular Android application with minSDK = API 22 Android 5.1 Lollipop.
• Then click File > New > Robot Application. Then notice these changes:

These two robot development related icons in Android Studio become enabled: Emulator and Connect/Disconnect. The 4th icon Wakeup is still grayed out since I don’t have a real robot to connect to.

Source: Margaret Maynard-Reid/Medium

The Android project structure gets updated to the Robot Project Structure. Dependencies are automatically added in build.gradle file:

compile 'com.aldebaran:libqi-java-android:sdk-2016-05-16'
compile 'com.aldebaran:qisdk:0.7'
compile 'com.aldebaran:qichatplayer:1.0.1'

• I set CPU/ABI as x86 and checked “Use Host GPU” under Run/Debug Configuration/AVD option. Make sure to launch the emulator by clicking on the Robot Emulator icon, instead of using a virtual device from the Android Studio AVD Manager.
• When the Robot emulator launches, the Robot Viewer also gets launched.

Without an actual robot I was still able to follow the first 3 of the tutorials: Say “Hello, world!”, Go to one meter forward and Mimic the elephant.

I will share more as I continue to learn about Pepper. Hopefully in the near future, we all can make apps that work with Pepper the humanoid robot.

Image: The Construct

By Miguel Angel

One of the most wanted robot simulations is a robot that can be used for anything. Robonaut is one like this. NASA kindly gave this simulation for public use and we thought here in The Construct that we could use it to make an even better user-friendly version. We created a test to demonstrate the possibilities that The Construct has to offer in Space Zero gravity simulations. We also envisioned this as a proof of concept of how The Construct could be used as a platform for competitions such as RoboCup, special for Space Exploration themes.

To launch this simulation you simply have to select the Robonaut2 simulation and select the following launch: $r2_robocup_at_space_1.launch

And there you have it! All the technical hassle was solved by The Construct and you have a ready to use Robonaut. This launch will spawn the torso version of Robonaut, with all its controllers in a zero gravity space. It will also spawn a taskboard full of functional buttons and switches without the third panel which will be spawned separately. One it’s done, you should get something like this:

Image: The Construct

To check that Robonaut is functioning correctly, it should have its arms in an L shape ready to start moving like in the picture above. Once all is running, please check that the taskboard is publishing. The taskboard publishes in a topic the state of the switches and buttons in the panel. In this early version, it only tracks the three lower switches and the security switch on the top right-hand corner of the first panel. To check all the taskboard systems are working, this is the correct output to the following command in the Webshell:

$rostopic info /taskboard/TaskboardPanelA

Type:
gazebo_taskboard/TaskboardPanelA
Publishers: * /taskboard_state_tracker
Subscribers: * /taskboard_systems_actuator

You should see the publisher of the Topic and a subscriber that controls when to activate the magnet that maintains the panel to be placed in position.

Image: The Construct

Image: The Construct

Image: The Construct

Image: The Construct

Once you are ready just launch the grab and place panel 3 launch:

Source: The Construct

This will reset the power switch and the panel to their starting positions.
Once done, Robonaut will start moving to grab the panel.

Source: The Construct

Source: The Construct

Source: The Construct

Once grabbed, it will place it just in front of the desired spot.

Source: The Construct

Then it will activate the magnet by firstly opening the security lid and then activating the power switches. Once activated, the panel will receive a force that will hopefully place it inside the taskboard.

Source: The Construct

This launch can be executed as many time as you like, so that there is no need of relaunching the simulation every time. There are other features that you might want to explore like the “model_destination_validator.py” which you can use to evaluate correct placing of objects based on their relative position to prior placed markers. Very useful for unsupervised learning and competitions.

Here is a full-length demo:

https://www.youtube.com/watch?v=QABa4pbk4qs

Come on, try it! We would love to see intelligent algorithms and systems to make Robonaut work as a charm!

There are magnificent tutorials about how to create plugins for Gazebo in the GazeboSim webpage. There are even some tutorials about how to create plugins for Gazebo + ROS, which show that there are several types of plugins (world, model, sensor, system, visual), and indicate how to create a plugin for a world-type plugin. But I recently I needed to create a plugin for a light detector and couldn’t find a concrete example. Here’s a how-to post showing you how I did it.

How to: light sensor plugin in Gazebo

Following the indications provided at the answers forum of Gazebo, I decided to build a very simple light detector sensor based on a camera. Instead of using a raytracing algorithm from lights, the idea is to use a camera to capture an image, then use the image to calculate the illuminance of the image, and then publish that illuminance value through a ROS topic.

Since the plugin is meant to be used with ROS, the whole plugin should be compilable using ROS environment.

Creating a ROS package for the plugin

First we need to create the package in our catkin workspace that will allow us to compile the plugin without a problem.

cd ~/catkin_ws/src
catkin_create-pkg gazebo_light_sensor_plugin gazebo_ros roscpp

Creating the plugin code

For this purpose, since we are using a camera to capture the light, we are going to create a plugin class that inherits from the CameraPlugin. The code that follows has been created taking as guideline the code of the authentic gazebo ROS camera plugin.

Create a file called light_sensor_plugin.h inside the include directory of your package, including the following code:

#ifndef GAZEBO_ROS_LIGHT_SENSOR_HH
#define GAZEBO_ROS_LIGHT_SENSOR_HH

#include <string>

// library for processing camera data for gazebo / ros conversions
#include <gazebo/plugins/CameraPlugin.hh>

#include <gazebo_plugins/gazebo_ros_camera_utils.h>

namespace gazebo
{
class GazeboRosLight : public CameraPlugin, GazeboRosCameraUtils
{
/// brief Constructor
/// param parent The parent entity, must be a Model or a Sensor
public: GazeboRosLight();

/// brief Destructor
public: ~GazeboRosLight();

/// brief Load the plugin
/// param take in SDF root element
public: void Load(sensors::SensorPtr _parent, sdf::ElementPtr _sdf);

/// brief Update the controller
protected: virtual void OnNewFrame(const unsigned char *_image,
unsigned int _width, unsigned int _height,
unsigned int _depth, const std::string &_format);

ros::NodeHandle _nh;
ros::Publisher _sensorPublisher;

double _fov;
double _range;
};
}
#endif

As you can see, the code includes a node handler to connect to the roscore. It also defines a publisher that will publish messages containing the illuminance value. Two parameters have been defined: fov (field of view) and range. At present only fov is used to indicate the amount of pixels around the center of the image that will be taken into account to calculate the illuminance.

Then create a file named light_sensor_plugin.cpp containing the following code in the src directory of your package:

#include <gazebo/common/Plugin.hh>
#include <ros/ros.h>
#include "/home/rtellez/iri-lab/iri_ws/src/gazebo_light_sensor_plugin/include/light_sensor_plugin.h"

#include "gazebo_plugins/gazebo_ros_camera.h"

#include <string>

#include <gazebo/sensors/Sensor.hh>
#include <gazebo/sensors/CameraSensor.hh>
#include <gazebo/sensors/SensorTypes.hh>

#include <sensor_msgs/Illuminance.h>

namespace gazebo
{
 // Register this plugin with the simulator
 GZ_REGISTER_SENSOR_PLUGIN(GazeboRosLight)

 ////////////////////////////////////////////////////////////////////////////////
 // Constructor
 GazeboRosLight::GazeboRosLight():
 _nh("light_sensor_plugin"),
 _fov(6),
 _range(10)
 {
 _sensorPublisher = _nh.advertise<sensor_msgs::Illuminance>("lightSensor", 1);
 }

 ////////////////////////////////////////////////////////////////////////////////
 // Destructor
 GazeboRosLight::~GazeboRosLight()
 {
 ROS_DEBUG_STREAM_NAMED("camera","Unloaded");
 }

 void GazeboRosLight::Load(sensors::SensorPtr _parent, sdf::ElementPtr _sdf)
 {
 // Make sure the ROS node for Gazebo has already been initialized
 if (!ros::isInitialized())
 {
 ROS_FATAL_STREAM("A ROS node for Gazebo has not been initialized, unable to load plugin. "
 << "Load the Gazebo system plugin 'libgazebo_ros_api_plugin.so' in the gazebo_ros package)");
 return;
 }

 CameraPlugin::Load(_parent, _sdf);
 // copying from CameraPlugin into GazeboRosCameraUtils
 this->parentSensor_ = this->parentSensor;
 this->width_ = this->width;
 this->height_ = this->height;
 this->depth_ = this->depth;
 this->format_ = this->format;
 this->camera_ = this->camera;

 GazeboRosCameraUtils::Load(_parent, _sdf);
 }

 ////////////////////////////////////////////////////////////////////////////////
 // Update the controller
 void GazeboRosLight::OnNewFrame(const unsigned char *_image,
 unsigned int _width, unsigned int _height, unsigned int _depth,
 const std::string &_format)
 {
 static int seq=0;

 this->sensor_update_time_ = this->parentSensor_->GetLastUpdateTime();

 if (!this->parentSensor->IsActive())
 {
 if ((*this->image_connect_count_) > 0)
 // do this first so there's chance for sensor to run once after activated
 this->parentSensor->SetActive(true);
 }
 else
 {
 if ((*this->image_connect_count_) > 0)
 {
 common::Time cur_time = this->world_->GetSimTime();
 if (cur_time - this->last_update_time_ >= this->update_period_)
 {
 this->PutCameraData(_image);
 this->PublishCameraInfo();
 this->last_update_time_ = cur_time;

 sensor_msgs::Illuminance msg;
 msg.header.stamp = ros::Time::now();
 msg.header.frame_id = "";
 msg.header.seq = seq;

 int startingPix = _width * ( (int)(_height/2) - (int)( _fov/2)) - (int)(_fov/2);

 double illum = 0;
 for (int i=0; i<_fov ; ++i)
 {
 int index = startingPix + i*_width;
 for (int j=0; j<_fov ; ++j)
 illum += _image[index+j];
 }

 msg.illuminance = illum/(_fov*_fov);
 msg.variance = 0.0;

 _sensorPublisher.publish(msg);

 seq++;
 }
 }
 }
 }
}

That is the code that calculates, in a very basic form, the illuminance.

Create a CMakeLists.txt

Copy the following code in your CMakeLists.txt

cmake_minimum_required(VERSION 2.8.3)
project(gazebo_light_sensor_plugin)

# Load catkin and all dependencies required for this package
find_package(catkin REQUIRED COMPONENTS
roscpp
gazebo_ros
)

# Depend on system install of Gazebo
find_package(gazebo REQUIRED)

link_directories(${GAZEBO_LIBRARY_DIRS})
include_directories(${Boost_INCLUDE_DIR} ${catkin_INCLUDE_DIRS} ${GAZEBO_INCLUDE_DIRS})

add_library(${PROJECT_NAME} src/light_sensor_plugin.cpp)
target_link_libraries(${PROJECT_NAME} ${catkin_LIBRARIES} ${GAZEBO_LIBRARIES} CameraPlugin)

catkin_package(
DEPENDS
roscpp
gazebo_ros
)

Update the package.xml

Now you need to include the following line in your package.xml, between the tags <export></export>

<gazebo_ros plugin_path="${prefix}/lib" gazebo_media_path="${prefix}" />

Now you are ready to compile the plugin. Compilation should generate the library containing the plugin inside your building directory.

Create a world file

Below is an example of a world file that includes the plugin. Save this code in a file named light.world inside the directory worlds of your package. This world file simply loads the camera with its plugin, so it might be a bit ugly, but it will be good enough for your tests. Feel free to add more elements and models in the world file.

<?xml version="1.0" ?>
<sdf version="1.4">
 <world name="default">
 <include>
 <uri>model://ground_plane</uri>
 </include>

 <include>
 <uri>model://sun</uri>
 </include>

 <!-- reference to your plugin -->
 <model name='camera'>
 <pose>0 -1 0.05 0 -0 0</pose>
 <link name='link'>
 <inertial>
 <mass>0.1</mass>
 <inertia>
 <ixx>1</ixx>
 <ixy>0</ixy>
 <ixz>0</ixz>
 <iyy>1</iyy>
 <iyz>0</iyz>
 <izz>1</izz>
 </inertia>
 </inertial>
 <collision name='collision'>
 <geometry>
 <box>
 <size>0.1 0.1 0.1</size>
 </box>
 </geometry>
 <max_contacts>10</max_contacts>
 <surface>
 <contact>
 <ode/>
 </contact>
 <bounce/>
 <friction>
 <ode/>
 </friction>
 </surface>
 </collision>
 <visual name='visual'>
 <geometry>
 <box>
 <size>0.1 0.1 0.1</size>
 </box>
 </geometry>
 </visual>
 <sensor name='camera' type='camera'>
 <camera name='__default__'>
 <horizontal_fov>1.047</horizontal_fov>
 <image>
 <width>320</width>
 <height>240</height>
 </image>
 <clip>
 <near>0.1</near>
 <far>100</far>
 </clip>
 </camera>
 <plugin name="gazebo_light_sensor_plugin" filename="libgazebo_light_sensor_plugin.so">
 <cameraName>camera</cameraName>
 <alwaysOn>true</alwaysOn>
 <updateRate>10</updateRate>
 <imageTopicName>rgb/image_raw</imageTopicName>
 <depthImageTopicName>depth/image_raw</depthImageTopicName>
 <pointCloudTopicName>depth/points</pointCloudTopicName>
 <cameraInfoTopicName>rgb/camera_info</cameraInfoTopicName>
 <depthImageCameraInfoTopicName>depth/camera_info</depthImageCameraInfoTopicName>
 <frameName>camera_depth_optical_frame</frameName>
 <baseline>0.1</baseline>
 <distortion_k1>0.0</distortion_k1>
 <distortion_k2>0.0</distortion_k2>
 <distortion_k3>0.0</distortion_k3>
 <distortion_t1>0.0</distortion_t1>
 <distortion_t2>0.0</distortion_t2>
 <pointCloudCutoff>0.4</pointCloudCutoff>
 <robotNamespace>/</robotNamespace>
 </plugin>
 </sensor>
 <self_collide>0</self_collide>
 <kinematic>0</kinematic>
 <gravity>1</gravity>
 </link>
 </model>
 </world>
</sdf>

Create a launch file

Now comes the final step: to create a launch that will upload everything for you. Save the following code as main.launch inside the launch directory of your package.

<launch>
 <!-- We resume the logic in empty_world.launch, changing only the name of the world to be launched -->
 <include file="$(find gazebo_ros)/launch/empty_world.launch">
 <arg name="verbose" value="true"/>
 <arg name="world_name" value="$(find gazebo_light_sensor_plugin)/worlds/light.world"/>
 <!-- more default parameters can be changed here -->
 </include>
 </launch>

Ready to run!

Now launch the world. Be sure that a roscore is running or your machine, and that the GAZEBO_PLUGIN_PATH environment var includes the path to the new plugin.

Now execute the following command:

roslaunch gazebo_light_sensor_plugin main.launch

You can see what the camera is observing by running the following command:

rosrun image_view image_view image:=/camera/rgb/image_raw

You can also have the value of luminance by watching the published topic:

rostopic echo /gazebo_light_sensor_plugin/lightSensor

Conclusion

Now you have a plugin for your Gazebo simulations that can measure (very roughly) the light detected. You can use it in you local copy of Gazebo or even inside The Construct.

Calculating kinematics is a cornerstone skill for robotics engineers.  Kinematics can sometimes be a pain, however, being told to “go and calculate the Forward Kinematics” is almost robotics research shorthand for “go and get familiar with this robot”. It’s the vital first step when using any new robot in research, particularly for manipulators.

Even though I had learned the theory of kinematics in university, it wasn’t until I had calculated various kinematic solutions for a few real research robots that the whole process started to feel intuitive. Even then, because I was not calculating kinematics every day I had to go back to my notes to “remind myself” how to do it every time. It would have been really helpful to have a step-by-step guide to remind myself which stages to go through, without having to read through hundreds of pages of academically written equations in text books. A sort of kinematics “cheat sheet” would have been useful.

This post is exactly that sort of cheat sheet. For now, we’ll primarily focus on the Devanit-Hartenberg (DH) approach to forward kinematics, as it’s the most common, and we’ll get to inverse kinematics in a future post.

Step 1: Get a Pencil and Paper

It can be tempting to jump straight for the computer when starting out with a new robot. However, even if the robot looks like a “standard” 6R manipulator, I always sit down with a pencil and paper to draw out the kinematic diagram. This simple task forces you to carefully consider the actual physical configuration of the robot, avoiding false assumptions that can wreak havoc later on during coding.

There are various ways to draw a kinematic chain. Pick whichever style you prefer. I favor simple cylinders for the revolute joints and lines for the links. Do a Google Image Search for “kinematic diagram” and see some of the different styles available.

As you draw, work out which way each joint moves and draw this motion as double-ended arrows onto the diagram.

 
Step 2: Figure Out Your Axes

The next key step is to draw the axes onto each joint. The DH approach assigns a different axis to each movable joint.

If you set up your axes correctly then working with the robot will be easy. Set them up incorrectly and you will suffer countless headaches. These axes will be required by simulators, inverse kinematic solvers, and even other researchers (nobody wants to solve a forward kinematic problem if someone else has already done it).

The z-axis should lie on the axis of rotation, for a revolute joint, or axis of extension, for a prismatic joint. The x-axis should lie along the “common normal”, which is the shortest orthogonal line between the previous z-axis and the current z-axis. Have a look at the video below to see how to set them up:

Personally, I draw the axes using the following coloring: z-axis (blue), x-axis (red) and y-axis (green). This is the coloring scheme that’s used in the RViz visualizer from ROS. Back in my undergraduate days, our lecturer made an axis “sculpture” out of three colored straws stuck into a sphere of blue-tack to explain the theory to us. Though this might seem a bit “playschool,” it can sometimes actually be helpful as you can position the sculpture next to the physical robot to make sure you’ve got the axes pointing in the right direction. For a virtual version of this, check out this interactive tool.

Step 3: Remember Your End Effector

The goal of calculating the forward kinematics is to be able to calculate the end effector pose from the position of the joints.

Most forward kinematic tutorials will generalize the end effector as a single distance from the final joint. This is fine for a simple “open-close” gripper. However, as modern grippers are often more complicated than this, it’s worth considering how the end effector operates. The Robotiq 3-Fingered Adaptive Gripper, for example, has a few different gripping modes. Each mode will correspond to a slightly different desired end effector pose. You should consider the end effector carefully when formulating the kinematic model.

Step 4: Calculate the DH parameters

Denavit-Hartenberg parameters are often required to enter the robot model into a simulator and start performing any sort of analysis on it.

The best way to visualize the DH parameters is to once again watch the video above. Have a look one more time with the DH parameters in mind.

The DH parameters break down each joint of the robot into four parameters, each taken with reference to the previous joint. They are calculated in reference to the “common normal” described above. Note that if the previous z-axis intersects the current z-axis, which is often the case, the common normal has a length of zero.

• d – the distance between the previous x-axis and the current x-axis, along the common normal
• θ – the angle around the z-axis between the previous x-axis and current x-axis.
• a (or r) – the length of the common normal
• α – the angle around the common normal to between the previous z-axis and current z-axis.

Go through each joint on your drawing and write down the DH parameters for each joint. Each joint should have one value, which is a variable representing the actuated joint. For a more detailed explanation and some examples, I recommend this handout by Peter Corke or this chapter from Introduction to Robotics.

Alternatives to the DH parameters — The DH approach is the most common approach to forward kinematics, but it’s not perfect. One of its failings is that it doesn’t handle parallel z-axes very elegantly. There are various alternatives, including Screw Theory representations, Hayati-Roberts and other geometric modelings (see this paper for a comparison). Although these may (or may not) be better approaches, most of the kinematic libraries below do accept the DH parameters, and for that reason it’s a reasonable approach to begin with.

Step 5: Choose Your Library

There are loads of kinematic software libraries, and many of them do far more than just calculate forward kinematics. Most of them include inverse kinematic solvers, dynamics, visualization, motion planning and collision detection, to name just a few features. These libraries will transform your DH parameters into matrices, which are then multiplied together to calculate the relationship between joint positions and end effector pose.

Of course, you can also quite easily “roll your own” forward kinematic library using your favorite programming language, if you’re inclined that way. I’ve taken this approach myself, but apart from being a good learning exercise it didn’t offer many more benefits than using an existing library.

Some good development libraries include: Robotics Library, Orocos Kinematics and Dynamics Library, ROS MoveIt, OpenRave, RoboAnalyzer and the Matlab Robotics Toolbox.

Even though you’ll usually require inverse kinematics to actually control the robot, computing the forward kinematics is a necessary step to get familiar with any new robotic arm. If you found this post useful, make sure to bookmark it so you can find it when you next encounter a new robot!

What is your favorite method of getting familiar with a new robot’s kinematics? Do you prefer another method over the DH parameters? Have you got any questions on implementing Forward Kinematics in your research robot? Tell us in the comments below.

Enjoy this post? Find more tutorials and robotics how-tos here. 

By Ilia Baranov

Robots are expensive and they usually depend on batteries. What if you want to run an experiment with 100 robots, running for 10 hours? To help answer this question, ROS has built in support for robot simulation in the form of Gazebo. While this works quite well, it is currently unable to simulate physical properties like batteries, temperature, surface roughness. If your robotics research depends on accurate models, you may want to consider looking at MapleSim® 2015 – a high performance physical modeling and simulation tool developed by Maplesoft.

Below we can see a video of the Grizzly RUV taking an open loop control path around a surface. Elements like current and voltage provided by the batteries, surface slipperiness and weight distribution all play a role in where the Grizzly actually ends up. To see what we’re referring to, watch the quick simulation video below:

https://youtu.be/dnhotIAyYBk

The MapleSim model features a 200 Ah lead-acid battery pack with a nominal voltage of 48 V, similar to the Type B battery pack used in the robot, to provide electrical power to move the vehicle on an uneven terrain. The lead acid battery used in the model is part of MapleSim’s Battery Component Library. The physical behaviors of the battery are described by mathematical expressions curve-fitted based on experiment measurements to provide the accurate battery voltage and state of charge during the operations of the robot.

The interaction forces and moments at the tire-terrain contact points are generated based on a 3D tire model. A 3D mathematical expression is used to describe the terrain surface to allow the tire-terrain contact points to be calculated based on the position of the vehicle. This 3D function is also used to generate the STL graphics of the terrain for animation (see Figure 1).

The model also outputs electric motor torques, speeds, and battery state of charge as shown below:

Setting up the simulation involves using the Grizzly RUV MapleSim model, available at the bottom of this post. The graphical representation is easy to understand and also quick to modify.

Maple and MapleSim provide a testing and analysis environment based on the virtual prototypes of the model. A number of analysis can be performed:

• Virtual testing and analysis: an engineer can easily test the operations of the robot for any design in a virtual environment through simulations in MapleSim. Using Maple, different terrain surface conditions and tire force models to fit different test scenarios can be generated. As an example, the plot below shows different battery energy consumption (state of charge) rates under different terrain conditions.

• Battery optimization: the developed MapleSim model can be coupled with Maple’s powerful optimization toolboxes to determine the optimal battery size and optimize the Battery Management System (BMS) to minimize energy consumption, reduce battery temperature, and increase battery service life.
• Motor sizing: the robot is equipped with four electric motors that are independently controlled to provide wheel torques and steering maneuvers. The seamless integration with Maple will allow the motor sizing optimization to be performed based on MapleSim simulations.
• Chassis design and payload distribution: the virtual prototype of the system will allow engineers to easily vary payload locations and distributions and analyze their effects, e.g., roll-over, stability, controllability, etc., on certain tasks.
• Path planning: Using Maple, different terrain surface conditions and tire force models to fit different test scenarios can be generated for path planning.
• Model-based controller design: the MapleSim model will allow the control strategies to be designed and tested for accuracy before being deployed on a real vehicle.
• Localization and mapping: the high-fidelity dynamic model of the robot will allow state estimation algorithms, such as Kalman filter and other Bayesian-based filtering algorithms, to be performed at a high accuracy.
• Optimized code generation: optimized C code can be generated from the MapleSim model for purpose of implementations of control, localization, and path planning strategies.

See here for more information on simulating the Grizzly in MapleSim.

In this new tutorial series, robotics educator and author Myke Predko introduces you to the basics of Arduino with simple projects that can be created in just a few minutes with inexpensive and easy-to-obtain parts. In this tutorial, Myke dives into how to program your Arduino so that you can plan an effective software architecture and monitor your application. Look for new tutorials every 2-3 weeks, or see all the tutorials in this series here. 

Welcome back. In this article, I want to discuss some of the particulars of programming an Arduino so that you better understand what is happening when your program is built. I’ll also discuss how to best architect your software. The Arduino software approach is somewhat unusual compared to other devices and, for beginners at least, takes away the need to do the high level design of the application. As a result, there are a few things to understand and keep in the back of your mind if you want your application to run without anything unexpected happening. In this article, I will provide you with a couple of simple applications to illustrate how Arduino software works, as well as a way to monitor the operation of the application.

In case you found the opening paragraph ominous and were wondering the Arduino is a potential minefield when it comes to software development, let me reassure you that it is as easy to program as any other device I’ve worked with and has been designed to allow for very sophisticated applications. It balances the needs of the beginner and expert application developer quite nicely.

When you’re creating with real time applications (applications that respond to different inputs), the common basic architecture is:

• System Initialization (“setup()”)
  • Define and initialize system and state variables
  • Setup hardware interfaces
• Repeat input/output processing code (“loop()”)
  • Get inputs
  • Process inputs with/update system and state variables
  • Drive outputs

The Arduino development software eliminates the need to come up with this program type, instead using the “setup()” and “loop()” methods, which provide this capability for you. As can be seen in Figure 1, when you first start the Arduino development tool or start a new “sketch” .ino (which I usually hear called an “ee-no”) file, the basic code with “setup()” and “loop()”are created – it is now up to you to put in the code for these two methods.

You may have heard that Arduino programming is really C++. However, when you compare example C++ programs to the Arduino .ino code, you will find that though there are some similarities, they really don’t look the same. This is because the .ino code is appended to the end of the C++ file containing the following “main()”:

#include <Arduino.h>

void setup();
void loop();

int main(void)
{
 setup();

#if defined(USBCON)
USBDevice.attach();
#endif

setup();
for (;;) {
  loop();
  if (serialEventRun) serialEventRun();
 }
 return 0;
}

// user .ino file is appended here

In the “main()” code, you can see the calls to the “setup()” and “loop()” methods, along with a few other pieces that need to be explained.

The first is the “#include <Arduino.h>” statement, in which the basic operations provided by the Arduino system are defined. “#include” statements copy in the contents of the specified file before the file is compiled.

Next are the “USBCON” and “USBDevice” statements. These are required if an Arduino USB Host Shield is used in the application. An Arduino USB Host Shield is a shield that enables the Arduino application to be augmented with USB hubs, keyboards, mice, joysticks, thumb drives and other devices.

We will be discussing the “for” statement and looping in a later article, but I wanted to point out that the “for(;;)” statement is a way of creating an infinite loop. When execution reaches this statement, the statements inside the curly braces (“{“ & “}”) that follow will be repeated as long as the Arduino is running.

I will discuss the “Serial Events” in a bit more detail below, but its purpose is to poll the Arduino’s serial communications hardware and perform any necessary operations. This is an important function for helping you to understand how your application is working.

Finally, the “return 0;” statement is put there as a matter of form – execution will never reach it because of the infinite loop above it. Many compilers will flag a warning or an error if a return statement is not at the end of your code, so putting it in will avoid this potential problem.

To demonstrate the operation of the “setup()” and “loop()” methods, I created the Console_Test application, which you can copy and paste into the Arduino development system:

// Console_Test - Message from "setup()" and "loop()"
//
// myke predko
void setup() {
 Serial.begin(9600);       // Initialize Serial Library
 Serial.println("Hello world!"); // Output with New Line
}

void loop() {
 Serial.println("Loop Line"); // Output with New Line
}

This application performs the following actions:

• Enables “Serial Events”, and outputs a message indicating that it is up and running
• Every time the “loop()” method is called, outputs a message

To test out this application, start up the Arduino development system and copy the following statements into “setup()”:

Serial.begin(9600);       // Initialize Serial Library
Serial.println("Hello world!"); // Output with New Line

and the following statement into “loop()”:

Serial.println("Loop Line"); // Output with New Line

Next, save the sketch as “Console_Test” and then transfer it to your Arduino using the “Upload” button shown in Figure 1. To see the application running and outputting the messages, click on the “Open Serial Monitor” button at the top right corner of Figure 1. After a few seconds, the Arduino will reset and restart running the application. You will see a display like Figure 2 for a second or so … at least until the repeating “Loop Line” messages cause the “Hello World!” message to scroll off the screen.

The “Serial.begin” statement loads in and starts (“instantiates” in Object Orientated speak) the “Serial” object, which is used for serial communications between the Arduino’s processor and the host PC. Serial communications are set at 9,600 bits per second (bps), which is actually 960 bytes per second (as ten bits are required for sending each byte of data). The “Serial.println” statements send the included string of characters to the host PC.

Serial Events are a great way to show that your application is working and to provide instructions to the Arduino during operation, but there are a couple of things to be wary of.

If you look at the “main()” code above, you will see that Serial Events only execute after a “loop()” operation. In the Arduino, Serial Events are not interrupt-driven; the receive buffer is only checked after a “loop()” and while the send buffer is being transmitted. The hardware receive registers in the ATMega, the processor used in the Arduino, can really only store one byte at a time safely, so the “loop()” code must execute in less time than it takes to receive a single byte (which is 10.4 milliseconds). This is not a major issue but it does mean that you should make your “loop()” code execute as quickly as possible – this is especially true if you work at higher speeds. Normally, I like to have communications at a minimum of 115,200 bps, which means that if I were to run at this communications speed in an Arduino, my “loop()” code must always execute in less than 87 microseconds to avoid the possibility of losing characters.

When you’re starting out, running Arduino “Serial” communications at 9,600 bits per second will probably be fast enough for most applications. As long as you don’t have long delays in your “loop()” code, you should not have any issues.

The second potential issue to watch for is the size of the “Serial” send and receive buffers, which are only 64 bytes in size. These are in software, and are independent of the hardware registers. Sending a message longer than 64 bytes will result in parts of it being “truncated” (cut off) and allowing the receive buffer to grow beyond 64 bytes will result in incoming data being lost. This may seem like an unlikely occurrence, but it can happen if all the bytes in the send message(s) in “setup()” are longer than 64 bytes, as they aren’t sent until “loop()” is first executed. It is possible to increase the size of the “Serial” send and receive buffers, but with a bit of care in your programs, there will be no need to.

While it is not important at this point in time, everything that is loaded into the Arduino is built during the transfer operation. This includes “Libraries” and any other code that you want to bring in to your Sketches. The advantage of doing this is to avoid the need for separate build processes; it also allows the code to be modified easily. You may notice that the time the Arduino development tool takes to compile your application seems unreasonably long – that is because it is going out and getting all possible library and include file code, and compiling it along with your application before building an executable with only the necessary pieces.

I realize that there is a lot of information here. It can all be boiled down to the following statements for the beginning user:

1. Application setup is put in “setup()”
2. Repeating code for the application is put in “loop()” and must execute as quickly as possible
3. “Serial” is used for providing information about the execution of a program

… but, I wanted to provide background information for individuals who may have had some programming experience on different platforms and wanted to understand how the Arduino works “behind the curtains”. For completely new users, I’m hoping that the information here will help you when you start doing your own applications and things go a bit wonky.

Example Programs:

• Console_Test

Next Article: Arduino Integer Data Types

Websites for Further Reading:

• http://arduino.cc/en/Hacking/BuildProcess “Aduino Build Process” explaining the nuts and bolts of how things go together
• https://code.google.com/p/arduino/wiki/BuildProcess A practical explanation of how the build process works
• http://arduino.cc/en/Hacking/Bootloader?from=Tutorial.Bootloader Arduino Bootloader description and information
• http://arduino.cc/en/reference/libraries Arduino Libraries
• http://www.instructables.com/id/HOW-TO-use-the-ARDUINO-SERIAL-MONITOR/ How-To use the Arduino Serial Monitor
• http://arduino.cc/en/Main/ArduinoUSBHostShield Arduino USB Host Shield

Most “getting started” Arduino tutorials start by getting to you build a flashing LED application. While the application I present here is similar, rather than just getting the Arduino’s built-in LED to flash as a “one off” application, we will be setting up the parts and tools for projects that we will implement in the future. While I provide you with the tools to create your first Arduino application, you’ll be preparing for creating your own Arduino-based robots down the line.

Before we get started, let’s talk about some basic electrical rules you will need to understand for this project. Chances are you already know them, but it never hurts to spend a minute and review:

1. Current flows from positive voltage (marked with red) to negative voltage (marked with black, and often referred to as “ground” or “Gnd”).
2. Current flows through an LED in only one direction (which causes it to light) and the negative connection is normally marked with a flat on its package.

As I go on with the article series, I will expand upon these electrical rules, but this is all you have to know for now. If you look at Figure 1 at the start of this article, you will see that positive voltage from the Arduino has a red connection, the negative (Gnd) has a black connection, and the LED has a flat side that indicates for the direction of current flow through the device.

The parts needed for this first iteration of your Arduino development station are:

• Arduino Uno (R3 or R3 SMD). When you start out, I recommend that you use the “R3” version with the long rectangular chip in a socket rather than the small, flat square chip. The socketed chip can be replaced if it is ever damaged and it is about $5.00 cheaper.
• Breadboard for quickly creating (and taking apart) your circuits.
• 5mm PTH red LED. This common LED consists of a red “bead” that lights when current passes through it. As noted above, current only flows through an LED in one direction with the negative connection marked with a flat on the side of the package.
• 100 Ohm, 1/4Watt resistor. This part reduces the current that will flow through the LED. While you can get a single 100 Ohm resistor, I recommend that you get a kit of resistors so that you have a stock of different values that can be applied in different circumstances.
• 22 gauge wire. This is for connections – it is worth purchasing a breadboard wiring kit with precut and stripped wires for the breadboard.

There are many places where these parts can be bought. At the end of the article, I have provided pointers to Sparkfun and their part numbers for the components listed above. Sparkfun is an excellent supplier and one that I feel confident recommending – but I should point out that you may want to explore local electronics shops in your area. None of the components are hard to get and chances are you can buy the components locally for about the same price without having to wait for shipping.

The “Breadboard” is a type of electronic circuit board in which wires and component leads can be pushed into (and pulled out of) to make simple circuits quickly. As can be seen in Figure 2, the middle holes are connected together vertically while the holes on the outside rows are connected horizontally. I place components in the central area and let the vertical connections connect components together while the horizontal connections are used for “power rails”. Note that I connect the ground power rails on both sides of the breadboard, but only provide power to one side; I will be discussing the reason for this in a future article.

The “5V” and “Gnd” connections from the Arduino to the Breadboard power rails are a convention that I will be using for future articles. This setup allows for fast wiring of application circuitry while allowing for high power devices to be supplied separately and not from the Arduino. Many Arduino applications use its power supply in ways that could lead to burning it out or worse. This wiring convention and reasons behind it will be discussed in future articles.

With the electronics parts together, there are a few tools that I recommend that you buy at this time:

• Plastic kit box to provide for quick clean up and transportation to other locations. This should have separate compartment for various parts and tools. I like the Stanley Sortmaster because the compartment separators can be set to allow storage of larger parts, and there is very little chance that the small parts in one compartment will find their way into another.
• 6’ A-B USB Cable.
• Needle nose pliers.
• Wire strippers. These can double as wire cutters.

All these components can be purchased at a local dollar store. The USB Cable really doesn’t belong as a “tool”, but I wanted to make the point of looking for it at a dollar store – it will be considerably cheaper at a dollar store than at a specialty electronics store.

With the parts and tools available, you can now wire the circuit as shown in Figure 1. The circuit is quite simple: an LED with 100 Ohm resistor is connected to the “5V” output of the Arduino and the Arduino’s pin “3”. When pin “3” outputs a “LOW” voltage, current flows through the LED and it lights. When pin “3” outputs a “HIGH” voltage, current no longer flows through the LED and it no longer lights. This action can be tested by connecting the Arduino to your PC using the USB power (which provides power to the circuit) followed by disconnecting the wire from the Arduino’s pin 3 (the LED’s negative (flat) side) and connecting it to the 5V rail or one of the two Gnd rails on the breadboard.

As a side point, I highly recommend designing your circuits so they can be tested easily without requiring any Arduino software. I guarantee you that it will save hours of wondering why Arduino code doesn’t seem to be working when the real problem is that there is something wrong with the electrical circuit.

With the application built, you’re now ready to create the Arduino code to test everything out. At the end of the article, I have provided the download page for the Arduino software development environment (often called “Arduino Software”) (referred to here as the “Arduino Sketchpad”) and there are specific instructions for installing the software. It can be loaded into Windows PCs, Macs or Linux systems, and just takes a few minutes to install.

The Arduino Sketchpad’s many options can be somewhat overwhelming to new users, and for that reason, I suggest that beginners focus on the five icons that are labeled in Figure 3; these are the only controls that will be required for quite a while.

When you have your Arduino Sketchpad up and running, cut and paste in the following program:

// flash - Flash LED on Pin 3
//
// myke predko


void setup() {
 pinMode(3, OUTPUT); // Make Pin 3 an Output
}


void loop() {
 digitalWrite(3, HIGH); // Pin 3 is high voltage (LED off)
 delay(250); // Wait 1/4s
 digitalWrite(3, LOW); // Pin 3 is low voltage (LED ON)
 delay(250); // Wait 1/4s
}

Then, click the “Save Sketch” icon to save the Sketch (which is what the Arduino application is called), and then click on the “Transfer to Arduino” (aka “Upload”). After a few seconds, you will get the “Done uploading” message and the LED should start flashing (if it doesn’t then use the hint above for debugging the application – chances are you have the LED orientated the wrong way or the resistor connected to the wrong power rail).

Now, you have a working Arduino development station. I know it doesn’t seem like much right now, but in the coming weeks, we’ll look at using this as a base for learning more about Arduino and creating your own robots.

Further Reading:

• http://arduino.cc/en/main/software Arduino Software Download Page
• http://forum.arduino.cc/index.php?board=2.0 Arduino “Installation & Troubleshoot” Forum
• http://www.ladyada.net/learn/arduino/lesson1.html First project tutorial
• http://electronicsclub.info/leds.htm Introduction to Light Emitting Diodes (LEDs)
• http://www.electronics-tutorials.ws/resistor/res_9.html Resistor Tutorial and Summary

Parts:

• https://www.sparkfun.com/ for parts:
  • https://www.sparkfun.com/products/11021 Arduino Uno – R3
  • https://www.sparkfun.com/products/9590 LED – Basic Red 5mm
  • https://www.sparkfun.com/products/10969 1/4W Resistor Kit
  • https://www.sparkfun.com/products/9567 Breadboard – Translucent
  • https://www.sparkfun.com/products/124 Jumper Wire Kit

Previous Article: Arduino for Makers #0: Introduction to the series
Next Article: Arduino Programing deconstructed

The tutorials take developers from the initial setup of TurtleBot through programming a web app to control it. The final demo is to bring coffee to people around an office, but my ultimate goal is for creative new users to learn from this simple idea and modify at their whim to generate inventive uses and expansions.

I produced this free series in hopes of bringing robotics technology to the web development community. As we recently saw with 3D printers, web developers have a long tradition of embracing emerging technologies. My goal, as a member of both the open-source robotics and web development communities, is to encourage web developers to start learning robotics so that we can see the same sort of creativity and popularity take off.

The way I see it, if we can give developers a taste of ROS’ power, they will invest the time to learn ROS properly. Today robotics stands on the foundation built by many brilliant people and engaged, innovative communities; now it is time to mine the untapped potential of developers not previously involved with robotics. By sparking developers’ interest and teaching them the basics of ROS, I hope to break down even more barriers and put us that much closer to making the robot revolution a reality.

The Arduino family of micro controllers has quickly become a go-to board for hobbyists due to its ease of use. Often times roboticists must create communication protocols to allow their embedded hardware to communicate with a computer. One of the most powerful aspects of ROS is that it can communicate with a large variety of hardware. That’s where rosserial comes in! Rosserial is a general protocol for sending ROS messages over a serial interface, such as the UART on Arduino. This allows you to easily interface any sensors attached to Arduino into your ROS environment. Read on to learn more!

This tutorial will get you started  by setting up the Arduino IDE and installing the rosserial libraries. Then, we’ll test it out by driving the Husky simulation using a neat trick with the Arduino board. It should goes without saying you’ll need an Arduino to complete this tutorial. We’ll also be using a Husky simulator, so make sure to run through our “Drive a Husky” tutorial if you haven’t done so yet.

Setup

The first step is to set up your Arduino environment, begin by installing the Arduino IDE and rosserial using:

sudo apt-get install arduino arduino-core ros-hydro-rosserial ros-hydro-rosserial-arduino

Once you open up the Arduino IDE for the first time, a directory named “sketchbook” will be created in your home directory. Now we need to import the rosserial library into your arduino IDE. If your sketchbook directory is empty, you may need to create a directory named “libraries” inside of it.

mkdir ~/sketchbook/libraries
cd ~/sketchbook/libraries
rosrun rosserial_arduino make_libraries.py .

Restart your Arduino IDE and you should see the ros_lib part of your libraries!

You’ll want to ensure that your Ubuntu user is part of the “Dailout” group, which grants you access to serial ports. You can check by using

groups "username”

If you don’t see “dailout”, you can easily add yourself using

sudo gpasswd --add "username" dialout

One last step to make our lives easier will be to create udev rules so your Arduino is  recognized when plugged in and the correct permissions are set. For more information on udev rules check out our udev article . Begin by plugging in your Arduino.

NOTE:If you are using a virtual machine, you will have to connect Arduino to the virtual machine after plugging it in.

You can confirm your system is actually connected to Arduino by running

ls -l /dev

Depending on your specific Arduino board, you should either see a line with ttyAMC# or ttyUSB#, where # is a number. This will also tell you the current permissions for your Arduino. Chances are you wouldn’t be able to upload code to your Arduino at the moment because you don’t sufficient permissions, but we will soon solve that by creating a new udev rule!

The udev rule will require the product and vendor ID to be able to identify when our arduino is connected which you can easily find using

lsusb

You should see a line similar to

Bus 002 Device 005: ID 2341:0043

To confirm this is indeed your Arduino, disconnect it and run the command again, taking note which entry has disappeared. Remember the ID numbers, in this case, 2341 is the vendor ID and 0043 is the product ID. Now venture over to your udev rules at:

cd /etc/udev/rules.d/

and create our new rules file, the naming conventions for rules files follows “##-name.rules”. Chose a number that isn’t in use!

sudo gedit 97-arduino.rules

Copy the following into your new rules file, replacing #### with your product ID and vendor ID. For more information about what these tags mean, check out our article on udev rules.

SUBSYSTEMS=="usb", ATTRS{idProduct}=="###", ATTRS{idVendor}=="####", SYMLINK+="arduino",MODE="0777",

All that is left is to update your udev rules and reboot your system

udevadm trigger
sudo reboot

You should now see “arduino” as an entry in ls -l /dev with full permissions! (rwxrwxrwx)

Code

We’re now set to upload our code to Arduino! The code is fairly straight forward, however if you have any difficulties following along, check out our “Creating a publisher” tutorial. Copy the following code into the Arduino IDE and click upload. If your udev rules were set correctly you should be able to upload without any errors.

If you encounter any errors, verify your arduino is coming up as “arduino” in a ls -l /dev and proper permissions are set. You may also have to point the Arduino IDE towards the correct USB port in tools -> serial port.

#include <ArduinoHardware.h>
#include <ros.h>
#include <geometry_msgs/Twist.h>

ros::NodeHandle nh;

geometry_msgs::Twist msg;

ros::Publisher pub("husky/cmd_vel", &msg);

void setup()
{
 nh.initNode();
 nh.advertise(pub);
} void loop()
{
 if(digitalRead(8)==1)
 msg.linear.x=-0.25;
 
 else if (digitalRead(4)==1)
 msg.linear.x=0.25;
 
 else if (digitalRead(8)==0 && digitalRead(4)==0)
 msg.linear.x=0;
 
 pub.publish(&msg);
 nh.spinOnce();
 }

Driving Husky

Now that Arduino is loaded with our code and publishing velocity commands, we can pass these messages along into our ROS environment. We’ll start by launching a Husky simulation:

roslaunch husky_gazebo husky_empty_world.launch

All that’s left is to attach the Arduino into our ROS environment using:

rosrun rosserial_python serial_node.py _port:=/dev/arduino

We’re ready to try it out! Go ahead and touch the digital pin 8 and you should see Husky drive forward! Similarly if you touch digital pin 4 Husky will drive backwards.

This trick is made possible by a phenomenon known as parasitic capacitance, which is usually an unwanted effect in electronics design, but serves nicely for the purpose of our example. That being said, this isn’t the most reliable method, and is intended to provide a simple example with minimal equipment.  If you are having difficulties moving your simulated Husky, try using rostopic echo /husky/cmd_vel to verify some commands are in fact being sent to Husky when you touch the pins.

Be sure to go through the rest of our ROS tutorials on our knowledge base. If you want to learn more about rosserial, be sure to visit the rosserial page of the ROS wiki.

Looking for other Clearpath tutorials? You might like these ones! Or visit www.support.clearpathrobotics.com.

In the previous ROS 101 post, we provided a quick introduction to ROS to answer questions like What is ROS? and How do I get started? Now that you understand the basics, here’s how they can apply to a practical example. Follow along to see how we actually ‘do’ all of these things …

First, you will need to run Ubuntu, and have ROS installed on it. For your convenience, you can download our easy-to-use image here:

https://s3.amazonaws.com/CPR_PUBLIC/LEARN_ROS/Learn_ROS-disk1.vmdk
https://s3.amazonaws.com/CPR_PUBLIC/LEARN_ROS/Learn_ROS.ovf

Login (username): user
Password: learn

Get VMWare Player, and use the virtual disk above. If you don’t want to use the provided image, follow the tutorial here (after installing Ubuntu 12.04):

http://wiki.ros.org/hydro/Installation/Ubuntu

Throughout the rest of the tutorials, we will be referencing the ROS cheatsheet available here.

Open a new Terminal Window (Ctrl + Alt + T).
In the new terminal, type roscore (And press enter).
This should produce something similar to the below.

What you have just done is started the ROS Master as we described above. We can now experiment with some ROS commands.
Open a new Terminal, and type in rostopic. This will give you a list of all the options that the rostopic command can do. For now, we are interested in rostopic list
Type in rostopic list. (And press enter). This should give you a window like the following:

The two entries listed above are ROS’s built in way of reporting and aggregating debug messages in the system. What we want to do is publish and subscribe to a message.
You can open a new Terminal again, or open a new tab on the same terminal window (Ctrl + Shift + T).
In the new Terminal, type in: rostopic pub /hello std_msgs/String “Hello Robot”

Let’s break down the parts of that command.
rostopic pub – This commands ROS to publish a new topic.
/hello – This is the name of the new topic. (Can be whatever you want)
std_msgs/String – This is the topic type. We want to publish a string topic. In our overview examples above, it was an image data type.
“Hello Robot” – This is the actual data contained by the topic. I.E. the message itself.
Going back to the previous Terminal, we can execute rostopic list again.
We now have a new topic listed! We can also echo the topic to see the message. rostopic echo /hello

We have now successfully published a topic with a message, and received that message.
Type Ctrl + C to stop echoing the /hello topic.
We can also look into the node that is publishing the message. Type in rosnode list. You will get a list similar to the one below. (The exact numbers beside the rostopic node may be different)

Because we asked rostopic to publish the /hello topic for us, ROS went ahead and created a node to do so. We can look into details of by typing rosnode info /rostopic_…..(whatever numbers)
TIP: In ROS, and in Linux in general, whenever you start typing something, you can press the Tab key to auto-complete it. If there is more than one entry, double tap Tab to get the list. In the above example, all I typed was rosnode info /rost(TAB)

We can get info on our topic the same way, by typing rostopic info /hello.

You will notice that the node listed under “Publishers:” is the same node we requested info about.

Up till now, we have covered the fundamentals of ROS, how to use rostopic, rosnode.

Next time, we will compile a short example program, and try it out.

See all the ROS101 tutorials here. If you liked this article, you may also be interested in:

• Up and flying with the AR.Drone and ROS: Getting started
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Clearpath Robotics brings us a new tutorial series on ROS!

Since we practically live in the Robot Operating System (ROS), we thought it was time to share some tips on how to get started with ROS. We’ll answer questions like where do I begin? How do I get started? What terminology should I brush up on? Keep an eye out for this ongoing ROS 101 blog series that will provide you with a top to bottom view of ROS that will focus on introducing basic concepts simply, cleanly and at a reasonable pace. This guide is meant as a groundwork for new users, which can then be used to jump into in-depth data at wiki.ros.org. If you are totally unfamiliar with ROS, Linux, or both, this is the place for you!

The ROS Cheat Sheet

This ROS Cheat Sheet is filled with tips and tricks to help you get started and to continue using once you’re a true ROS user. This version is written for ROS Hydro Medusa. Download the ROS Cheat Sheet here.

What is ROS?

ROS (Robot Operating System) is a BSD-licensed system for controlling robotic components from a PC. A ROS system is comprised of a number of independent nodes, each of which communicates with the other nodes using a publish/subscribe messaging model. For example, a particular sensor’s driver might be implemented as a node, which publishes sensor data in a stream of messages. These messages could be consumed by any number of other nodes, including filters, loggers, and also higher-level systems such as guidance, pathfinding, etc.

Why ROS?

Note that nodes in ROS do not have to be on the same system (multiple computers) or even of the same architecture! You could have a Arduino publishing messages, a laptop subscribing to them, and an Android phone driving motors. This makes ROS really flexible and adaptable to the needs of the user. ROS is also open source, maintained by many people.

General Concepts

Let’s look at the ROS system from a very high level view. No need to worry how any of the following works, we will cover that later.

ROS starts with the ROS Master. The Master allows all other ROS pieces of software (Nodes) to find and talk to each other. That way, we do not have to ever specifically state “Send this sensor data to that computer at 127.0.0.1. We can simply tell Node 1 to send messages to Node 2.

How do Nodes do this? By publishing and subscribing to Topics.

Let’s say we have a camera on our Robot. We want to be able to see the images from the camera, both on the Robot itself, and on another laptop.

In our example, we have a Camera Node that takes care of communication with the camera, a Image Processing Node on the robot that process image data, and a Image Display Node that displays images on a screen. To start with, all Nodes have registered with the Master. Think of the Master as a lookup table where all the nodes go to find where exactly to send messages.

In registering with the ROS Master, the Camera Node states that it will Publish a Topic called /image_data (for example). Both of the other Nodes register that they are Subscribed to the Topic /image_data.

Thus, once the Camera Node receives some data from the Camera, it sends the /image_data message directly to the other two nodes. (Through what is essentially TCP/IP)

Now you may be thinking, what if I want the Image Processing Node to request data from the Camera Node at a specific time? To do this, ROS implements Services.

A Node can register a specific service with the ROS Master, just as it registers its messages. In the below example, the Image Processing Node first requests /image_data, the Camera Node gathers data from the Camera, and then sends the reply.

We will have another tutorial “ROS 101 – Practical Example” next week.

See all the ROS101 tutorials here. If you liked this article, you may also be interested in:

• Up and flying with the AR.Drone and ROS: Getting started
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

If that’s the case, or if you’re simply interested in learning more about the Thymio, head over to Thymio & Aseba. There you can find an excellent guide on using the visual programming language (VPL) bundled with Aseba, to program the Thymio II. This guide includes an introduction to the Thymio II and the VPL, and contains a number of small projects of increasing complexity, designed to gradually introduce new features, sensors, actuators and ideas to the reader.

On the Thymio & Aseba site, you will also find numerous project examples (for example ball balancing, as pictured above), many of which include Aseba source code examples and an implementation discussion. You can also find various constructions, which take advantage of the Thymio’s LEGO Technic interoperability to build many weird and wonderful creations including cranes, funicular railways, walking robots and more.

Of these many project examples, three projects really stood out to me:

1. The simple pendulum, where the Thymio’s accelerometer is used to investigate the dynamics of a swinging pendulum;
2. The scales (shown below), where mechanical scales are built and the Thymio’s distance sensor used to digitize the weight measurement; and
3. The reflectivity measure, which uses the Thymio’s IR sensor to measure optical reflectivity of various objects at known distances, automated using the Thymio’s wheels to drive an arm mechanism.

These three examples demonstrate that robots, although cool in their own right, are also an excellent tool for learning more about the world around us.

This is the third tutorial in the Up and flying with the AR.Drone and ROS series.

In this tutorial we will:

1. Learn about the AR.Drone’s state feedback (and how it is handled by ROS)
2. Learn about the AR.Drone’s tag detection
3. Program our first ROS nodes: A subscriber and a publisher

In the previous tutorials we:

1. Installed ROS, the AR.Drone driver and AR.Drone keyboard controller then flew the AR.Drone using the provided keyboard controller (link)
2. Learned about ROS communication, setup and then flew the AR.Drone using a joystick (link)

In the next tutorials, we will:

• Write a controller to enable tag following
• Write a controller for the drone which allows us to control drone velocity, rather than body angle

What you’ll need for the tutorial

1. An AR.Drone!
2. A computer or laptop with WiFi
3. Linux, either the ARDroneUbuntu virtual machine, or a custom installation
4. Either the keyboard or joystick controller from the previous tutorials

I also assume that you’ve completed either the keyboard or joystick tutorial, which detail how to setup the software, connect to the drone, etc, and provide a method of controlling the AR.Drone from your computer.

Updating

If you have not completed the joystick tutorial:

Please go back and update your installation according to the instructions listed there.

If you downloaded the ARDroneUbuntu virtual machine:

To update your installation for this tutorial, click the “Update Tutorials” button in the ARDroneUbuntu Virtual Machine

If you use a custom installation:

At the terminal, enter the following:

$ roscd ardrone_autonomy
$ git pull
$ roscd ardrone_tutorials
$ git pull
$ roscd
$ rosmake ardrone_tutorials

AR.Drone State Feedback – A First Look

Preparation:

1. Find the Orange-[Green|Yellow|Blue]-Orange tag that came with the AR.Drone.
2. In your ARDroneUbunutu virtual machine (or custom installation), open a new terminal window and type:

   $ roscd ardrone_tutorials/launch

3. If you intend to fly with the keyboard, type:

   $ geany keyboard_controller_with_tags.launch

   Otherwise, if you intend to fly with the joystick:

   $ geany joystick_controller.launch

4. Now find the line:

   <param name="enemy_colors" value="3" />

5. And change the value to “1”, “2”, or “3”, depending on whether your tags are Orange-Green-Orange, Orange-Yellow-Orange or Orange-Blue-Orange respectively.

Tag Detection and Feedback Data

1. Run the system (but don’t takeoff with the drone!) using either the keyboard_controller:

   $ roslaunch ardrone_tutorials keyboard_controller_with_tags.launch

   or the joystick_controller:

   $ roslaunch ardrone_tutorials joystick_controller.launch

2. Open a new terminal window and type:

   $ rostopic list

   • This list shows all the active ROS “topics” used for communication to and from the AR.Drone
3. In the same terminal window, type:

   $ rostopic echo /ardrone/navdata

   • This echoes the AR.Drone’s feedback data into the terminal window.
   • Note that many items will be zero because the AR.Drone is not yet flying
4. Move the Orange-Color-Orange tag in front of the AR.Drone’s front camera. It should be detected, and the console output (from the previous step), should update to reflect this detection.
   • Note the tags_xc and tags_yc values. These represent the x/y location of the tag within the image.
   • The video display window will show a dot at these coordinates, and the distance measurement above the dot
5. Now place the tag somewhere that can be seen by the flying drone. After this is done, take off with the drone.
6. Watch the console output. What happens if you fly forwards? sideways? What variable changes if you rotate the drone?

Plotting the AR.Drone’s State

1. A more intuitive way of representing this feedback data is using a plot. PressCtrl-C to terminate the echo process from the last stage, and then type:

   $ rxplot /ardrone/navdata/rotX,/ardrone/navdata/rotY

   • This launches a plotting window showing the AR.Drone’s estimated roll (rotX) and pitch (rotY).
   • Similar plots could, for example, be generated for yaw (rotZ), x and y velocity (vx and vy), altitude (altd), etc.
2. Similarly to displaying the AR.Drone’s feedback data, we can use the plotting method to investigate how we are controlling the drone:

   $ rxplot /cmd_vel/linear/y,/cmd_vel/linear/x

   • This plots the roll (linear/y) and pitch (linear/x) commands.
   • Note that these are scaled with a value of $\pm1$ being the maximum commandable angle (set by the euler_angle_max in the launch file).
   • Note also that the direction of AR.Drone movement is commanded (ie. move along the x-axis = /cmd_vel/linear/x = commanded pitch), whereas the AR.Drone’s feedback data is named with reference to axis of rotation (ie. tilt around y-axis = /ardrone/navdata/rotY = feedback pitch).

Logging Data

1. To log data, we can use the inbuilt ROS command rosbag:

   $ rosbag record -O ARDroneFlight.bag /ardrone/navdata /cmd_vel

   • This will record the topics /ardrone/navdata and /cmd_vel to a “bag-file”, stamped with the current date and time.
   • Leave this process running for a few seconds, fly the AR.Drone around, and when finished press Ctrl-C to terminate.
   • Recording all topics is possible with rosbag record -a, however this also records the AR.Drone’s HD video stream, thus bag-files will quickly become very large
   • The advantage of using rosbag to log data, is that bag-files can be replayed at a later time, and interact very nicely with the existing ROS infrastructure.
2. Land the AR.Drone and close any windows / terminate any processes relating to the controller. Now, into the terminal window that ran the AR.Drone controller, type:

   $ roscore

   Now open a new terminal window and type:

   $ rxplot /cmd_vel/linear/y,/cmd_vel/linear/x

   Now finally (ignoring the warning from the previous step), switch back to the terminal window that you used to run the rosbag command and type:

   $ rosbag play ARDroneFlight.bag

   Does the flight look familiar? It should! This is the flight you recorded in the last step! Use Ctrl-C to terminate the above processes.

Custom Data Handling

In the last section, we investigated the AR.Drone’s feedback data by using ROS’s inbuilt plotting (rxplot) and logging (rosbag) commands. In this section, you will learn how we get this feedback data into your own application by writing a custom ROS Node. This node will be used to compare the commanded roll and pitch angles with the AR.Drone’s roll and pitch estimates, thus laying the foundations for the next tutorial, where we will use this feedback data to programmatically control the AR.Drone.

Messages, Topics and Nodes

In ROS terminology, a program is known as a Node. To interact with other nodes (for example the AR.Drone driver), each node will have some subscribers, which listen for information messages on a specific topic (communications channel); and some publishers, which send messages to a specific topic.

These messages can be anything. An example that we saw in the previous section was the Navdata message, which is sent over the /ardrone/navdata topic and contains feedback data from the AR.Drone.

We will now investigate this message a little further. Open up a new terminal window and type

rosmsg show ardrone_autonomy/Navdata

You should be presented with the following output:

$ rosmsg show ardrone_autonomy/Navdata
std_msgs/Header header
  uint32 seq
  time stamp
  string frame_id
float32 batteryPercent
uint32 state
int32 magX
int32 magY
int32 magZ
int32 pressure
int32 temp
float32 wind_speed
float32 wind_angle
float32 wind_comp_angle
float32 rotX
float32 rotY
float32 rotZ
int32 altd
float32 vx
float32 vy
float32 vz
float32 ax
float32 ay
float32 az
uint32 tags_count
uint32[] tags_type
uint32[] tags_xc
uint32[] tags_yc
uint32[] tags_width
uint32[] tags_height
float32[] tags_orientation
float32[] tags_distance
float32 tm

This listing shows us the various members of the Navdata message, including their datatypes. Note that the Navdata message has a header, which includes the time the message is sent.

Minimum Viable Subscriber

Open up a new terminal window and type:

$ roscd ardrone_tutorials
$ geany src/minimum_viable_subscriber.py

Up will pop a geany window, ready for your to edit the new file. Enter the following into the new geany window, then save and close the file:

import roslib
import rospy

roslib.load_manifest('ardrone_tutorials')

from ardrone_autonomy.msg import Navdata

def ReceiveData(data):
  print '[{0:.3f}] Pitch: {1:.3f}'.format(data.header.stamp.to_sec(), data.rotY)

rospy.init_node('minimum_viable_subscriber')
sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)

while not rospy.is_shutdown():
  pass

We will now use the previously recorded bag-file to test the node that we’ve just written.

1. In one terminal window, enter:

   $ roscore

   Which begins the roscore process, which coordinates the interactions between nodes.

2. Now in a new terminal window type:

   $ roscd ardrone_tutorials
   $ python src/minimum_viable_subscriber.py

   This will launch the node that we’ve just programmed.

3. Finally, in a third terminal window, begin the bag-file playback with:

   $ rosbag play ARDroneFlight.bag

   • Note you must be in the same directory as the bag-file for the above command to work
4. If you switch back to the terminal window in which our new node is running, you should see a stream of time-stamped printouts with the drone’s pitch, for example:

   [1353596639.491] Pitch: 4.375
   [1353596639.511] Pitch: 4.128
   [1353596639.531] Pitch: 3.750
   [1353596639.552] Pitch: 3.304
   [1353596639.571] Pitch: 3.063

Minimum Viable Subscriber: A Closer Look

If you made it through the previous section, congratulations, you have successfully written and tested a ROS node, which listens for Navdata measurements from the AR.Drone (or equivalently: a recorded flight) and outputs a time-stamped pitch measurement. But how does it all work?

Lets take a look at the source-code from the previous section, line by line:

import roslib
import rospy

With these two lines, we import the standard ROS packages, required for every ROS node.

roslib.load_manifest('ardrone_tutorials')

Once the required packages are imported, load the manifest file. This file details the dependencies of the current project (for example, that ardrone_tutorials is dependent on the AR.Drone driver, ardrone_autonomy), which through the above command are loaded into the path.

from ardrone_autonomy.msg import Navdata

Once the project dependencies have been loaded, we can import those which are needed by the current node. In the above case, we load the Navdata message definition from the ardrone_autonomy package.

def ReceiveData(data):
  print '[{0:.3f}] Pitch: {1:.3f}'.format(data.header.stamp.to_sec(), data.rotY)

Here we define a callback function to handle the Navdata messages. This function prints a formatted string containing the message’s timestamp and the AR.Drone’s pitch angle. This callback function will be used later in the program.

rospy.init_node('minimum_viable_subscriber')

Before issuing any commands to the rospy package, we need to initialize a ROS node.

sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)

We can now subscribe the node to various topics. Above we subscribe the node to the /ardrone/navdata topic, which carries Navdata messages. Received messages should then be handled by the previously-defined ReceiveData function.

while not rospy.is_shutdown():
  pass

Because our program is relying on callbacks for data processing, we need to stop the program from terminating. The above defines a loop that runs until the node is shut down (for example with Ctrl-C). In most situations the above loop would perform some sort of processing, however in our example, we simply pass (do nothing).

Publishing Data

In this section we will introduce the concept of publishing data to a topic (the opposite of subscribing). We will be using this more extensively in the next tutorial, where we will be programmatically controlling the AR.Drone by publishing messages to the /cmd_vel topic.

Minimum Viable Publisher

The goal of this section is to create a minimum_viable_publisher.py file such that Navdata messages are averaged over a second and the average pitch published to the (new) /ardrone/average_pitch topic. Modify the file as follows:

import roslib
import rospy

roslib.load_manifest('ardrone_tutorials')

from ardrone_autonomy.msg import Navdata
from std_msgs.msg import String

messages = []

def ReceiveData(data):
  messages.append(data)

rospy.init_node('minimum_viable_publisher')
sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)
pub_Average = rospy.Publisher('/ardrone/average_pitch', String)

r = rospy.Rate(1)
while not rospy.is_shutdown():
  if len(messages)>0:
    avg = sum([m.rotY for m in messages])/len(messages)
    messages = []

    avgmsg = String()
    avgmsg.data = 'Average Pitch: {0:.3f}'.format(avg)
    pub_Average.publish(avgmsg)
  r.sleep()

You can run the program using the following series of commands:

1. In one terminal, run:

   $ roscore

2. In the next terminal, run the new program

   $ roscd ardrone_tutorials
   $ python src/minimum_viable_publisher.py

3. We now echo the published data to the topic we’ve created:

   $ rostopic echo /ardrone/average_pitch

4. And finally, like we’ve done twice before, we begin the playback of our recorded flight

   $ rosbag play ARDroneFlight.bag

5. Switching back to the terminal in which we ranrostopic echo, you should see a new message with the average pitch appearing once a second:

   data: Average Pitch: 0.649
   ---
   data: Average Pitch: 1.078
   ---
   data: Average Pitch: 3.781
   ---
   data: Average Pitch: 3.285
   ---

Minimum Viable Publisher: A Closer Look

Comparing minimum_viable_publisher.py to minimum_viable_subscriber.py from the last section, we can see many similarities:

from ardrone_autonomy.msg import Navdata
from std_msgs.msg import String

We additionally import the String message, required for our new topic.

messages = []
def ReceiveData(data):
  messages.append(data)

We have updated the navdata callback function to save the messages in a queue, rather than process them straight away.

Note: because we’re accessing the messages list in both a callback function and the main loop, we should be using Python’s threading.Lock to control access to the list and prevent race conditions. See ardrone_tutorials/src/drone_video_display.py for an example.

pub_Average = rospy.Publisher('/ardrone/average_pitch', String)

We create a new publisher, responsible for publishing messages of the String type to the /ardrone/average_pitch topic. Note that we do not need to explicitly initialize this topic, it will be initialized automatically on first usage.

The biggest change is in the program’s main loop, which I will now cover in a little more detail:

r = rospy.Rate(1)

Firstly we define a Rate object. We use this to control the frequency of the main processing loop. In our case, we want to print a message every second, thus we want a frequency of $1 \text{Hz}$.

while not rospy.is_shutdown():
  if len(messages)>0:
    avg = sum([m.rotY for m in messages])/len(messages)
    messages = []

    avgmsg = String()
    avgmsg.data = 'Average Pitch: {0:.3f}'.format(avg)
    pub_Average.publish(avgmsg)

If messages have been received, we calculate the average pitch using a python list comprehension to extract the pitch (.rotY) from every Navdata message. We then empty the messages queue.

We then construct a new String message and set its data property, before publishing it to the topic we previously created.

r.sleep()

Finally, we tell the rate object to sleep. This command will calculate how long it needs to sleep, such that the desired loop frequency is achieved.

Conclusions

I hope you’ve enjoyed working through this tutorial and learning about data flow in the ROS environment. In the next tutorial, which I will release in a month’s time, we will use what we have learned today to programmatically control the AR.Drone, while at the same time learning about PID control.

If you have any questions about the tutorial, please leave me a comment below and I will get back to you as soon as possible!

You can see all the tutorials in here: Up and flying with the AR.Drone and ROS.

If you liked this article, you may also be interested in:

• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Classroom robotics: Motivating independent learning and discovery

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

This is the second tutorial in the Up and flying with the AR.Drone and ROS series.

In this tutorial we will:

1. Talk about the ROS communication hierarchy
2. Setup a joystick to work with ROS
3. Fly the AR.Drone with a joystick

In the previous tutorial we:

1. Installed ROS, the AR.Drone driver and AR.Drone keyboard controller then flew the AR.Drone using the provided keyboard controller (link)

In the next tutorials, we will:

• Look more closely at the data sent back from the drone and play with the on-board tag detection (link)
• Write a controller for the drone to enable tag following
• Write a controller for the drone which allows us to control drone velocity, rather than body angle

What you’ll need for the tutorial

To complete this tutorial you will need:

1. An AR.Drone!
2. A computer or laptop with WiFi
3. Linux, either the ARDroneUbuntu virtual machine, or a custom installation
4. A  (linux compatible) USB joystick or control pad

I also assume that you’ve completed the previous tutorial, which details how to setup the software, connect to the drone, etc.

Updating

It has been three weeks since the launch of the first tutorial and I am very happy with the response so far. The tutorial has been viewed over 720 times and ARDroneUbuntu downloaded over 50 times. I was also given the honor of partaking in RobotGarden’s very first Quadrocoptor Workshop, where we worked through the tutorial (and ironed out the bugs) together.

This response has taught me a few things about how to structure a tutorial series, and how to release a virtual machine. Based on what I have learned, I will be restructuring the source-code for the following tutorials. I will also be updating the virtual machine to reflect these changes.

When writing the first tutorial, I thought it would be better to have a separate repository for each tutorial to keep everything atomic and isolated to ensure the addition of a new tutorial would not affect anything in the past – However having learned from the video-hangout with RobotGarden, I realize this could cause an update nightmare down the track. For this reason, I have decided to operate from a single repository and will maintain a stable branch for each tutorial, with the master branch always synchronized with the stable version of the latest tutorial. The source-code for each tutorial will also contain all source-code required to run the previous tutorials.

What this means for you:

• If you downloaded the ARDroneUbuntu virtual machine before the release of this tutorial, you will need to click the “Update Tutorials” button twice. After this, your virtual machine will be updated to reflect the new changes.
• If you installed the tutorials manually, you will need to run the following at the terminal:

  # roscd
  # rm -rf ardrone_tutorials_getting_started
  # git clone https://github.com/mikehamer/ardrone_tutorials

• If you have not yet downloaded the VM, no problems! I have released an up-to-date version that you can download from here

Sorry for any hassle caused by the above, I hope this new structure will allow me to more easily deliver the next tutorials!

So without further ado, lets check out some data!

1. Connect to the AR.Drone as detailed in the first tutorial (remembering to disconnect other network adaptors if they use the 192.168.1.x address range)
2. From within your Linux installation, launch two terminal windows (ARDroneUbuntu users, refer to the first tutorial if you’re not sure what I mean)
3. In the first window, launch the KeyboardController from the first tutorial, using the command:

   $ roslaunch ardrone_tutorials keyboard_controller.launch

4. Wait for everything to load and the video display to become active, then in the second terminal window enter the command:

   $ rostopic list

   This command lists all active “topics”, the communication channels through which ROS programs (called “nodes”) communicate.

5. To view the currently running ROS Nodes, type:

   $ rxgraph

   The window that appears will show a network of nodes, and how they are connected through the topics that we saw in the previous step.

6. When you’re ready, close this window. We’re now going to have a look at the messages that are being sent between nodes. At the terminal, type:

   $ rostopic echo /ardrone/navdata

   The navdata message is published by the AR.Drone driver at a rate of 50Hz.

7. Press Ctrl-C to stop echoing the navdata message then switch to the other terminal window and press Ctrl-C again, to stop the roscore program

So what are we setting up today?

After that whirlwind tour of the ROS communication hierarchy, lets take a look at what we’ll be building today:

As shown above, today we’ll be setting up the joy_node (a standard ROS package) to listen for input from a joystick or control-pad. This input will be converted into a ROS message, then sent to our controller over the joy topic.

Our controller subscribes to the joy topic and reacts to the received messages by translating joystick movement into drone movement commands, which are then published to the cmd_vel topic; and button presses to takeoff, land, or emergency commands, which are each published to a respective channel.

The ardrone_driver receives these various commands and then sends them via the wireless network to the physical drone, which in turn sends back a video stream. This video stream is published by the ardrone_driver onto the /ardrone/image_raw topic, from where it is received and subsequently displayed by our controller.

Setting up a joystick

Firstly, we need to connect the joystick and set it up to work with ROS.

1. Open a terminal window (or reuse one from before), then type:

   $ ls /dev/input

2. Take note of the jsX devices (where X is a number). These are not your joystick.
3. Now connect your joystick to the computer and again type:

   $ ls /dev/input

4. If you see a new jsX device appear, congratulations, you’ve successfully connected the joystick, take note of which device this is, then progress to the next section of the tutorial
5. If no new jsX device appears when you connect the joystick, you’re most likely using a virtual machine and may need to add the USB device manually.
6. If you are using the ARDroneUbuntu virtual machine, open the Machine > Settings window, then the Ports > USB tab. With the joystick plugged in, add a new USB device. Your joystick should appear in the dropdown list.
7. Click OK, then unplug the joystick from the computer
8. Repeat steps 3 and 4, plugging the joystick into a different USB port if it is not detected in step 4.

Testing the joystick

Now that we have the joystick installed, let’s check to make sure everything is working. Type the following at a terminal, substituting jsX for the joystick device you previously identified:

$ jstest /dev/input/jsX

If you are greeted with a “jstest: Permission denied” error, type:

$ sudo chmod a+r /dev/input/jsX

Which enables all users to read the joystick device (ARDroneUbuntu users, the password is: ardrone). After running jstest, moving the joystick should cause the numbers on the screen to update, if not, try running jstest for the other jsX devices. Congratulations, your joystick device is working! Press Ctrl-C to exit.

Using the joystick with ROS

Now we’re ready to integrate the joystick device with the ROS ecosystem using ROS’s “Joy” package.

1. Open up three terminal windows. In the first, launch roscore:

   $ roscore

2. In the second, we launch ROS’s joystick launcher, replacing jsX with the joystick device you previously identified:

   $ rosparam set joy_node/dev "/dev/input/jsX"
   $ rosrun joy joy_node

3. Now in the third terminal window, we echo the Joy messages, sent out by the joy_node that we just launched:

   $ rostopic echo joy

4. If all is working, you should see a Joy message print on the console whenever the joystick is moved (or a button is pressed).

Identifying Control Buttons & Axes

We now have the task of identifying the various axis and buttons used to fly the drone. All in all, you will need to identify axes to control the AR.Drone’s roll, pitch, yaw-velocity and z-velocity; and buttons to trigger an emergency, command a takeoff, and command a landing.

To identify the emergency button for example, press the button you would like to trigger an emergency, and watch the terminal to see which button is enabled. In my case, I use the joystick’s trigger button for emergency (being able to react quickly to an emergency is very important). With the button pressed, the following is printed in the terminal window:

axes: [-0.0, -0.0, -0.0, -0.0, 0.0, 0.0]
buttons: [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

From this I can identify that: Emergency = Button 0 (note that indexing starts from zero). The process follows similarly for identifying an axis:

1. Press the joystick fully left. The axis that changes  (the first number in the list is axis 0) is the roll axis, the number if changes to is your roll scale (usually + or -1)
2. Press the joystick fully forward. The axis that changes is your pitch axis, and the number it changes to is the pitch scale (usually + or -1)
3. Rotate your joystick fully counter clockwise, the axis that changes is your yaw axis and the number it changes to is the yaw scale (usually + or -1)
4. Push your throttle stick fully forwards. The axis that changes is your z-axis and the number it changes to is the z-scale (usually + or -1).

axes: [1.0, -0.0, -0.0, -0.0, 0.0, 0.0]
buttons: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

From this I identify that the Roll Axis = 0 and Roll Scale = 1.

After you have identified all axes and buttons, press Ctrl-C in all terminal windows to stop the processes.

Configuring the Joystick Controller

In ROS, a “Launch File” is used to configure and launch a set of nodes. We’ve already seen this in the first tutorial (and at the beginning of this one), where we typed:

$ roslaunch ardrone_tutorials keyboard_controller.launch

In this command, keyboard_controller.launch configures the ARDrone driver for indoor flight, then launches both the driver and keyboard controller. Similarly in this tutorial, we will be launching the joystick controller (and related nodes) using a launch file.

Go ahead and open a terminal window, then type:

$ roscd ardrone_tutorials
$ geany launch/joystick_controller.launchtemplate

This will open the joystick_controller.launch file in the text editor (non ARDroneUbuntu users can use a text editor of choice). You will need to customize this file for your setup.

• Update line 21 to reflect your joystick device (the /dev/input/jsX that you identified earlier).
• Update lines 27-29 to reflect your button mapping
• Update lines 32-35 to reflect your axis mapping
• Update lines 38-41 to reflect your scales (usually either 1 or -1)
  These scaling factors determine for example the drone roll direction when the joystick is tilted along the roll axis. If your drone does the opposite of what you expect, change the sign on the relevant scaling factor (eg 1 becomes -1)

Feel free to have a look at some of the parameters used on the drone. The drone’s control parameters can potentially be increased to allow for more aggressive flight.

Once you’re happy with the settings, save this file as joystick_controller.launch, then close the geany application.

Finally flying!

So, after that rather long endeavor (which you should only need to do once), lets fly!

Connect to the drone, disconnect other network devices, then open a terminal and type:

$ roslaunch ardrone_tutorials joystick_controller.launch

This will launch the nodes as you configured in launch file. If all goes well, you should now be able to fly the drone using your joystick.

Before commanding the drone to take-off, you should first check that your emergency button works. With the drone still landed, press the emergency button once. All onboard LEDs should turn red. Pushing the button again will reset the drone, and the LEDs will once again turn green. In addition to this, whenever a (mapped) button is pressed, the joystick_controller will print a message to the terminal, such as “Emergency Button Pressed”. This is a further indication that things are working!

You’re now ready to take off. Have fun!

Going Further

The above child’s play? Heres some ideas for exploring the ROS and ARDrone environment:

• While the joystick controller is running, open a new terminal window and type:

  $ rxgraph

  This will display the node communication diagram that we investigated at the start of the post

• Take a look at the data being sent back from the drone:

  $ rostopic echo /ardrone/navdata

• Plot your flight speeds (or any other data of interest) with:

  $ rxplot /ardrone/navdata/vx,/ardrone/navdata/vy

• Adjust drone parameters in the joystick_controller.launch file to fly more aggressively

Troubleshooting

• The joystick doesn’t do anything
  Check that the joystick device you configured in the launch file corresponds to your joystick. To do this, repeat the section “Testing the Joystick”
• My joystick device is correct, but the joystick doesn’t do anything
  Assuming you’ve gone through the “Testing the Joystick” section successfully, launch the joystick controller again and in a separate terminal, type:

$ rostopic echo joy

Assuming everything is configured correctly, this will print joystick messages to the terminal when the joystick is removed.

• If nothing is printed, check to make sure you’ve configured the correct joystick device in the launch file, and that the joystick permissions are correct (see “Testing the Joystick”).
• If something is printed, check to make sure your axis and button mappings in the launch file are correct. If this is the case, leave a comment to this tutorial and I will be happy to help!

• The drone does the opposite of what I expect
  For example flying backwards when the joystick is tilted forwards. To solve this, determine the axes that respond counterintuitively then invert the sign of the respective scaling factors in the launch file. For example, if the pitch axis (forward-backward) is the opposite of what you would expect, the ScalePitch parameter would change from 1 to -1.
• I cannot connect to the drone
  Check that your computer is connected to the drone via wireless, and that all other network devices are disconnected
• The drone will not take off

• Are all the onboard LEDs red? If so, try pressing the emergency button. Do they toggle to green? Perfect – you’re ready to fly!
• If the LEDs are green and the drone still won’t take off, check that your button mapping is correct in the launch file

• Other issues?
  If you have any questions, or comments about what you would like to see in the next tutorials, please send me a message or leave a comment to this post!

You can see all the tutorials in here: Up and flying with the AR.Drone and ROS.

If you liked this article, you may also be interested in:

• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Classroom robotics: Motivating independent learning and discovery

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.