In this episode, Audrow Nash speaks to Dan O’Mara, who is the founder and COO of Circuit Launch and Mechlabs. Circuit Launch is a space for hardware entrepreneurs to work in Oakland, California, and Mechlabs is a project-based course to learn robotics. This interview is mostly about Mechlabs, but talks about the origins of Circuit Launch, including how it is not a maker or coworking space and its business model. For Mechlabs, we talk about several of its aspects that make it different than a university education in robotics, including how there are mentors not instructors, how projects are scoped, and how people are invited to work on what is most interesting to them. We also talk about the future of Mechlabs and how it fits with current universities.

Episode Links

• Download the episode
• Mechlabs’ website
• Circuit Launch’s website
• Dan’s LinkedIn

Podcast info

• Podcast website
• Apple Podcasts
• Spotify
• RSS
• Full episodes
• Clips

Do you fancy making yourself an industrial robot to enjoy at home? Jeremy Fielding, a passionate fan of mechanical engineering, did. So he built one. Good news is: he’s preparing a series of videos to teach you the whole process from scratch. How much power do you need to run 7 motors at one time? If you lose power, how do you prevent the arm from collapsing on you or dropping the load? How do you keep the cost down? He’s recorded over 100 hours of video, and he’s planning to teach you how he used the servo motors, how you can use them for your projects and how he designed his 7 axis, articulated robot.

Jeremy’s aim (website, YouTube, Twitter, Instagram) is simple: draw people to engineering with amazing projects, inspire them with ideas, then teach them how to do it. And for this video series, he’s also looking for your collaboration. So if you’ve got experience and knowledge on building this type of robots and you’d like to share it, maybe you end up being part of the series!

We’d like to thank Black in Robotics for making it possible for us to discover Jeremy. Here’s the video Jeremy has released to introduce his project:

We had to go find this guy on YouTube because he looked very cool! https://t.co/xomix1TZXo @jeremy_fielding #BlackInRobotics #NationalRoboticsWeek @RoboWeek

— Black in Robotics (@BlackInRobotics) April 9, 2021

Check out Jeremy’s YouTube channel to discover many more instructional videos. You can also support his work on Patreon.

In this episode, Audrow Nash speaks with Ian Bernstein, Founder and Head of Product at Misty Robotics, about a robotics platform designed for developers called Misty II.  Bernstein discusses the motivation behind making a robotics platform for developers (relating it to personal computers), Misty II’s hardware extensibility and software “skills,” and the future direction of Misty Robotics.

A video introducing the Misty II platform:

Ian Bernstein

Ian Bernstein is Founder and Head of Product at Misty Robotics, a spin-off company from Sphero, Inc. focused on building personal robots for the home and office. In this role, Bernstein leads Misty Robotics’ product development and design. Prior to Misty Robotics, Bernstein served as Founder and Chief Technology Officer at Sphero, Inc. that has shipped more than 3 million robots to date. Bernstein joined TechStars in 2010 with Sphero co-founder Adam Wilson and created Sphero, the original app-enabled robotic ball. Bernstein has had a lifelong passion for robotics and creation and has been building robots since he was 12. Ian holds a BS in Electrical and Electronics Engineering from Colorado State University.

Links

• Download mp3 (13.8 MB)
• Misty Robotics’ homepage
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash interviews Elliott Rouse, Assistant Professor at the University of Michigan, about an open-source prosthetic leg—that is a robotic knee and ankle. Rouse’s goal is to provide an inexpensive and capable platform for researchers to use so that they can work on prostheses without developing their own hardware, which is both time-consuming and expensive. Rouse discusses the design of the leg, the software interface, and the project’s timeline.

Elliott Rouse

Elliott Rouse is an Assistant Professor in the Mechanical Engineering Department at the University of Michigan, where he directs the Neurobionics Lab. The vision of his group is to discover the fundamental science that underlies human joint dynamics during locomotion and incorporate these discoveries in a new class of wearable robotic technologies. The Lab uses technical tools from mechanical and biomedical engineering applied to the complex challenges of human augmentation, physical medicine, rehabilitation and neuroscience. Dr. Rouse and his research have been featured at TED, on the Discovery Channel, CNN, National Public Radio, Wired Magazine UK, Business Insider, and Odyssey Magazine.

Links

• Rouse’s lab website
• Rouse’s personal website
• Download mp3 (12.6 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS
• Support us on Patreon

In this episode, Audrow Nash interviews Peter Corke, Professor of Robotics at the Queensland University of Technology and Director of the Australian Centre for Robotic Vision, about Robot Academy. Robot Academy is an online platform that provides free-to-use undergraduate-level learning resources for robotics and robotic vision.

The content was developed for two 6-week Massively Open Online Courses (MOOCs) that Corke taught in 2015 and 2016. This content is now available as individual lessons (over 200 videos, each less than 10 minutes long) or in masterclasses (collections of videos, around 1 hour in duration, previously a MOOC lecture). Unlike a MOOC, all lessons are available all the time.

While the content is typically designed for undergraduate-level students, around 20% of the lessons require no more than general knowledge. Each lesson is rated in terms of difficulty (on a 5-point scale), and Robot Academy references videos on Khan Academy to help students get up to speed to follow more advanced lessons.

Peter Corke

Peter Corke is Professor of Robotics and Control at the Queensland University of Technology leading the ARC Centre of Excellence for Robotic Vision in Australia. Previously he was a Senior Principal Research Scientist at the CSIRO ICT Centre where he founded and led the Autonomous Systems laboratory, the Sensors and Sensor Networks research theme and the Sensors and Sensor Networks Transformational Capability Platform. He is a Fellow of the IEEE. He was the Editor-in-Chief of the IEEE Robotics and Automation magazine; founding editor of the Journal of Field Robotics; member of the editorial board of the International Journal of Robotics Research, and the Springer STAR series. He has over 300 publications in the field and has held visiting positions at the University of Pennsylvania, University of Illinois at Urbana-Champaign, Carnegie-Mellon University Robotics Institute, and Oxford University.

Links

• Robot Academy’s homepage
• Peter Corke’s website
• Download mp3 (12.5 MB)
• Subscribe to Robots using iTunes
• Subscribe to Robots using RSS

What do you get when you put together wood and rope? Well according to Plymouth University’s Professor Guido Bugmann: a low-cost, open source, 2 meter tall robot! All buildable for under £2000. The Cheap Arm Project (CHAP) began as an MSc project aimed at developing an affordable mobile robot arm system that could be used by wheelchair users to access daily objects at inaccessible heights or weights (the extreme case being 2 litre bottle).

Now the project has turned open source in the hope of continuing development and gathering more feedback. With the entire hardware and software details available online, with detailed instructions, the team hopes that community members will attempt to construct their own CHAP robot.

While building an entire robot may conventionally seem a complex and expensive undertaking, CHAP stands out with its use of 3D printed parts and wood. In fact, the “under £2000” price tag of required parts is primarily made up of the use six Dynamixel motors, each costing up to £200 and used by the project due to having numerous available during development. With this in mind, future work will look at the use of lower cost motors.

Rather than a robot ‘brain’, CHAP is powered by a human one. To accomplish this, CHAP’s Raspberry Pi was used to host a local web page to enable tele-operation from any connected device, all accessible through a WiFi hotspot also hosted by the Pi. This was an important design choice as it removed the need for platform specific apps and associated costs, simultaneously simplifying deployment and testing.

This was done deliberately to keep the design simple, and to allow users the freedom to interact with their world. Early user-tests discovered that some level of intelligence would be needed to simplify controls. With currently three joints, and three axis available; some steps are needed to automate object interaction (pull requests welcome!).

The next step for CHAP will be refinement of hardware and software; however the very first will be building a community! With full design documents available it is hoped that with support these can be improved and simplified, while simultaneously improving the effectiveness of the software.

Link to the github site can be found here: https://mobile-chap.github.io/Web/

Figure 15: Adding 40m overshot to both ends of each survey line

UgCS is easy-to-use software for planning and flying UAV drone-survey missions. It supports almost any UAV platform, providing convenient tools for areal and linear surveys and enabling direct drone control. What’s more, UgCS enables professional land survey mission planning using photogrammetry techniques.

How to plan photogrammetry mission with UgCS

Standard land surveying photogrammetry mission planning with UgCS can be divided into following steps :

1. Obtain input data
2. Plan mission
3. Deploy ground control points
4. Fly mission
5. Image geotagging
6. Data processing
7. Map import to UgCS (optional)

https://www.youtube.com/watch?v=SalCQoG6WUw&feature=youtu.be

Step one: Obtain input data

Firstly, to reach the desired result, input settings have to be defined:

• Required GSD (ground sampling distance – size of single pixel on ground),
• Survey area boundaries,
• Required forward and side overlap.

GSD and area boundaries are usually defined by the customer’s requirements for output material parameters, for example by scale and resolution of digital map. Overlap should be chosen according to specific conditions of surveying area and requirements of data processing software.

Each data processing software (e.g., Pix4D, Agisoft Photoscan, Dronedeploy, Acute 3d) has specific requirements for side and forward overlaps for different surfaces. To choose correct values, please refer to documentation of chosen software. In general, 75% forward and 60% side overlap will be a good choice. Overlapping should be increased for areas with small amount of visual cues, for example for deserts or forests.

Often, aerial photogrammetry beginners are excited about the option to produce digital maps with extremely high resolution (1-2cm/pixel), and to use very small GSD for mission planning. This is very bad practice. Small GSD will result in longer flight time, hundreds of photos for each acre, tens of hours of processing and heavy output files. GSD should be set according to the output requirements of the digital map.

Other limitations can occur. For example, GSD of 10cm/pixel is required, but designed to use a Sony A6000 camera. Based on mentioned GSD and camera’s parameters, the flight altitude would be set to 510 meters. In most countries, maximum allowed altitude of UAV’s (without special permission) is limited to 120m/400ft AGL (above ground). Taking into account the maximum allowed altitude, the maximum possible GSD in this case could be no more than 2.3cm.

Step two: Plan your mission

Mission planning consists of two stages:

• Initial planning,
• Route optimisation.

-Initial planning:

The first step is to set surveying area using UgCS Photogrammetry tool. Area can be set using visual cues on underlying map or using exact coordinates of edges. The result – survey area is marked with yellow boundaries (Figure 1).

Figure 1: Setting the survey area

The next step is to set GSD and overlapping for the camera in Photogrammetry tool’s settings window (Figure 2).

Figure 2: Setting camera’s Ground Sampling Distance and overlapping

To take photos in Photogrammetry tool’s setting window, define the control action of the camera (Figure 3). Set camera by distance triggering action with default values.

Figure 3: Setting camera’s control action

At this point, initial route planning is completed. UgCS will automatically calculate photogrammetry route (see Figure 4).

Figure 4: Calculated photogrammetry survey route before optimisation

-Route optimisation

To optimise the route, it’s calculated parameters should be known: altitude, estimated flight time, number of shots, etc.

Part of the route’s calculated information can be found in the elevation profile window. To access the elevation profile window (if it is not visible on screen) click the parameters icon on the route card (lower-right corner, see Figure 5), and from the drop-down menu select show elevation:

Figure 5: Accessing elevation window from Route cards Parameters settings

The elevation profile window will present an estimated route length, duration, waypoint count and min/max altitude data:

Figure 6: Route values in elevation profile window

To get other calculated values, open route log by clicking on route status indicator: the green check-mark (upper-right corner, see Figure 7) of the route card:

Figure 7: Route card and status indicator, Route log

Using route parameters, it can be optimised to be more efficient and safe.

-Survey line direction

By default, UgCS will trace survey lines from south to north. But, in most cases, it will be more optimal to fly parallel to the longest boundary line of the survey area. To change survey line direction, edit direction angle field in the photogrammetry tool. In the example, by changing angle to 135 degrees, the number of passes is reduced from five (Figure 4) to four (Figure 8) and route length is 1km instead of 1.3km.

Figure 8: Changed survey line angle to be parallel to longest boundary

-Altitude type

UgCS Photogrammetry tool has the option to define how to trace the route according to altitude, with constant altitude above ground (AGL) or above mean sea level (AMSL). Please refer to your data processing software requirements as to which altitude tracking method it recommend.

In the UgCS team’s experience, the choice of altitude type depends on desired result. For orthophotomap (standard aerial land survey output format) it is better to choose AGL to ensure constant GSD for the entire map. If the aim is to produce DEM or 3D reconstruction, use AMSL so the data processing software has more data to correctly determine ground elevation by photos in order to provide more qualitative output.

Figure 9: Elevation profile with constant altitude above mean sea level (AMSL)

In this case, UgCS will calculate flight altitude based on the lowest point of the survey area.

If AGL is selected in photogrammetry tool’s settings, UgCS will calculate the altitude for each waypoint. But in this case, terrain following will be rough if no “additional waypoints” are added (see Figure 10).

Figure 10: Elevation profile with AGL without additional waypoints

Therefore, if AGL is used, add some “additional waypoints” flags and UgCS will calculate a flight plan with elevation profile accordingly (see Figure 11).

Figure 11: Elevation profile with AGL with additional waypoints

-Speed

In general, if flight speed is increased it will minimise flight time. But high speed in combination with large camera exposure can result in blurred images. In most cases 10m/s is the best choice.

-Camera control method

UgCS supports 3 camera control methods (actions):

1. Make a shot (trigger camera) in waypoint,
2. Make shot every N seconds,
3. Make shot every N meters.

Not all autopilots support all 3 camera control options. For example (quite old) DJI A2 does support all three options, but newer (starting from Phantom 3 and up to M600) cameras support only triggering in waypoints and by time. DJI promised to implement triggering by distance, but it’s not available yet.

Here are some benefits and drawbacks for all three methods:

Table 1: Benefits and Drawback for camera triggering methods

In conclusion:

• Trigger in waypoints should be preferred when possible
• Trigger by time should be used only if no other method is possible
• Trigger by distance should be used when triggering in waypoints is not possible to use

To select triggering method in UgCS Photogrammetry tool accordingly, use one of three available icons:

• Set camera mode
• Set camera by time
• Set camera by distance

-Glibal control

Drones, e.g., DJI Phantom 3, Phantom 4, Inspire, M100 or M600 with integrated gimbal, have the option to control camera position as part of an automatic route plan.

It is advisable to set camera to nadir position in the first waypoint, and in horizontal position before landing to prevent lenses from potential damage.

To set camera position, select the waypoint preceding the photogrammetry area and click set camera attitude/zoom (Figure 12) and enter “90” in the “Tilt” field (Figure 13).

Figure 12: Setting camera attitude

Figure 13: Setting camera position

As described previously, this waypoint should be a Stop&Turn type, otherwise the drone could skip this action.

To set camera to horizontal position, select last waypoint of survey route and click set camera attitude/zoom and enter “0” in the “Tilt” field.

-Turn types

Most autopilots or multirotor drones support different turn types in waypoints. Most popular DJI drones have three turn-types:

• Stop and Turn: drone flies to the fixed point accurately, stays at that fixed point and then flies to next fixed point.
• Bank Turn: the drone would fly with constant speed from one point to another without stopping.
• Adaptive Bank Turn: It is almost the same performance like Bank Turn mode (Figure 13), but the real flight routine will be more accurately than Bank Turn.

It is advisable not to use Bank Turn for photogrammetry missions. Drone interprets Bank Turns as “recommendation destination waypoint”. The drone will fly towards this direction but will almost never pass through the waypoint. Because drone will not pass the waypoint, no action will be executed, meaning the camera will not be triggered, etc.

Adaptive Bank Turn should be used with caution because a drone can miss waypoints and, again, no camera triggering will be initiated.

Figure 14: Illustration of typical DJI drone trajectories for Bank Turn and Adaptive Bank Turn types

Sometimes, adaptive bank turn type has to be used in order to have shorter flight time compared to stop and turn. When using adaptive bank turns, it is recommended to use overshot (see below) for the photogrammetry area.

-Overshot

Initially overshot was implemented for fixed wing (airplane) drones in order to have enough space for manoeuvring a U-turn.

Overshot can be set in photogrammetry tool to add an extra segment to both ends of each survey line.

Figure 15: Adding 40m overshot to both ends of each survey line

In the example (Figure 15) can be seen that UgCS added 40m additional segments to both ends of each survey line (comparing to Figure 8).

Adding overshot is useful for copter-UAVs in two situations:

1. When Adaptive Bank Turns are used (or similar method for non-DJI drones), adding overshot will increase the chance that drone will precisely enter survey line and camera control action will be triggered. UgCS Team recommends to specify overshot that is approximately equal to distance between the parallel survey lines.
2. When Stop and Turn type is in use in combination with action to trigger camera in waypoints, there is a possibility that before making the shot, drone will start rotation to next waypoint – it can result in having photos with wrong orientation or blurred. To avoid that, shorter overshot has to be set, for example 5m. Don’t specify too short value (< 3m) because some drones could ignore waypoints, that are too close.

Figure 16: Example of blurred image taken by drone in rotation to next waypoint

-Takeoff point

It is important to check the takeoff area at site before flying any mission! To better explain best practice on how to set takeoff point, first discuss an example of how it should not be done. Supposing that the takeoff point in our example mission (Figure 17) would be from the point marked with the airplane-icon, and drone pilot would upload the route on the ground with set automatic mission for automatic take-off.

Figure 17: Take-off point example

Most drones in automatic takeoff mode would climb to low altitude about 3-10meters and then fly straight towards the first waypoint. Other drones would fly towards first waypoint straight from ground. Looking closely at the example map (Figure 17), some trees between the takeoff point and the first waypoint can be noticed. In this example, the drone more likely will not reach a safe altitude and will hit the trees.

Not only the surroundings can affect takeoff planning. Drone manufacturers can change drones elevation behavior in drone firmware, therefore after firmware updates it is recommended that you check drones automatic takeoff mode.

Also, a very important consideration is that most small UAVs use relative altitude for mission planing. Altitude counted relatively according to first waypoint is a second reason why an actual takeoff point should be near the first waypoint, and on the same terrain level.

UgCS Team recommends placing the first waypoint as close as possible to actual takeoff point and specifying a safe takeoff altitude (≈30m in most situations will be above any trees, see Figure 18). This is the only method that warrants safe takeoff for any mission. It also protects from any weird drone behaviour and unpredictable firmware updates, etc.

Figure 18: Route with safe take-off

-Entry point to the survey grid

In the previous example, (see Figure 18), it can be noticed, that after adding the takeoff point, the route’s survey grid entry point was changed. This is because if additional waypoint is added subsequently to the photogrammetry area, UgCS will plan to fly the survey grid starting from nearest corner to the previous waypoint.

To change the entry point to survey grid, set additional waypoint close to the desired starting corner (see Figure 19).

Figure 19: Changing survey grid entry point by adding additional waypoint

-Landing point

If no landing point will be added outside the photogrammetry area after the survey mission, the drone will fly and hover in the last waypoint. There are two options for landing:

1. Take manual control over the drone and fly to landing point manually,
2. Activate the Return Home command in UgCS or from Remote Controller (RC).

In situations when the radio link with the drone is lost, for example if the survey area is large or there are problems with the remote controller, depending on the drone and it’s settings, one of these actions can occur:

• Drone will return to home location automatically if lost radio link with ground station,
• Drone will fly to last waypoint of survey area and hover as long as battery capacity will enable that, then: drone will perform emergency landing, or it will try to fly to home location.

The recommendation is to add an explicit landing points to the route in order to avoid relying on unpredictable drone behavior or settings.

If the drone doesn’t support automatic landing, or the pilot prefers to land manually, place the route’s last waypoint over the planned landing point with an altitude for comfortable manual drone descending and landing above any obstacles in the surrounding area. In general 30m is best choice.

-Action execution

Photogrammetry tool has a magic parameter “Action Execution” with three possible values:

• Every point
• At start
• Forward passes

This parameter defines how and where camera actions specified for photogrammetry tool will be executed.

The most useful option for photogrammetry/survey missions is to set forward passes, the drone will make photos only on survey lines, but will not make excess photos on perpendicular lines.

-Complex survey areas

UgCS enables photogrammetry/survey mission planning for irregular areas, having functionality to combine any number of photogrammetry areas in one route, avoiding splitting the area in separate routes.

For example, if a mission has to be planned for two fields connected in a T-shape, and if these two fields are marked as one photogrammetry area, the whole route will not be optimal, regardless any direction of survey lines.

Figure 20: Complex survey area before optimisation

If the survey area is marked as two photogrammetry areas within one route, survey lines for each area can be optimised individually (see Figure 21).

Figure 21: Optimised survey flight passes for each part of a complex photogrammetry area

Step three: deploy ground control points

Ground control points are mandatory if the survey output map has to be precisely aligned to coordinates on Earth.

There are lots of discussions about the necessity of ground control points in cases when a drone is equipped with Real Time Kinematics (RTK) GPS receivers with centimeter-level accuracy. This is useful, but the drone coordinates are not in themselves sufficient because, for precise map aligning, image center coordinates are necessary.

Data processing softwares like Agisoft Photoscan, Dronedeplay, Pix4d, Icarus OneButton and others will produce very accurate maps using geotagged images, but the real precision of the map will not be known without ground control points.

Conclusion: ground control points have to be used to create survey-grade result. For a map with approximate precision, it is sufficient to rely just on RTK GPS and the capabilities of data processing software.

Step four: fly your mission

For carefully planned missions, flying it is the most straightforward step. Mission execution differs according to the type of UAV and equipment used, therefore it will not be described in detail in this article (please refer to equipment’s and UgCS documentation).

Important issues before flying:

• In most countries there are strict regulations for UAV usage. Always comply with the regulations! Usually these rules can be found on web-site of local aviation authority.
• In some countries special permission for any kind of aerial photo/video shooting is needed. Please check local regulations.
• In most cases missions are planned before arriving to flying location (e.g., in office, at home) using satellite imaginary from Google maps, Bing, etc. Before flying always check actual circumstances at the location. There could be a need to adjust take-off/landing points, for example, to avoid tall obstacles (e.g., trees, masts, power lines) in your survey area.

Step five: image geotagging

Image geotagging is optional if ground control points were used, but almost any data processing software will require less time to process geotagged images.

Some of the latest and professional drones with integrated cameras can geotag images automatically during flight. In other cases images can be geotagged in UgCS after flight.

Very important: UgCS uses the telemetry log from drone, that is received via radio channel, to extract the drone’s altitude for any given moment (when pictures were taken). To geotag pictures using UgCS, assure robust telemetry reception during flight.

For detailed information how to geotag images using UgCS refer to UgCS User Manual.

Step six: data processing

For data processing, use third party software or services available on the market.

From UgCS Team experience, the most powerful and flexible software is Agisoft Photoscan (http://www.agisoft.com/), but sometimes too much user input is required to get necessary results. The most uncomplicated solution for users is online service Dronedeploy (https://www.dronedeploy.com/). All other software packages and services will fit somewhere between these two in terms of complexity and power.

Step seven (optional): import created map to UgCS

Should the need arise for the mission to be repeated in the future, UgCS enables importing the GeoTiff file as a map layer and using it for mission planning. More detailed instructions can be found in UgCS User Manual. See the result of an imported map created using UgCS photogrammetry tool imported as GeoTiff file in Figure 22.

Figure 22: Imported GeoTiff map as layer. The map is output of a Photogrammetry survey mission panned with UgCS

Visit the UgCS homepage

Download this tutorial as a PDF

If you liked this tutorial, you may also enjoy these:

• Programming for robotics: Introduction to ROS
• Python programming your NAO robot
• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Adjust how you learn and quickly pick up robotics programming
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

The NAIST OpenHand M2S was developed by a team of students as part of the school’s annual CICP project (read the blog post about it here), in which students can propose and organize their own research projects. Based on the Yale OpenHand M2, the NAIST OpenHand M2S was developed for textile manipulation, sensitive grasping as well as pinpoint pushing with high loads. All of the parts (except motors, sensors etc.) can be downloaded from here and 3D printed.

For grasping and manipulation, the hand is equipped with two 3D force sensors that act as fingertips. Using these, the hand can gently grasp textiles and let them glide through its fingers, pull them taut when tension is required, and even recognize different materials. By rubbing its fingertips together, a force signal is generated which allows the hand to sense if it successfully grasped a textile, and which kind of material it is.

Lastly, the rigid finger of the hand allows it to push with high loads and tuck into small openings. This can be used for tasks like making a bed, but also during the manufacturing of a car or airplane seat, which is part of the team’s research. Future versions of the hand will be able to pick up thin objects from a table easily without moving the robot.

You can read the conference paper here. Download the 3D files here, to use the gripper in your own research.

If you liked this article, you may also enjoy these:

• Robotics, maths, python: A fledgling computer scientist’s guide to inverse kinematics
• Python programming your NAO robot
• Soft robotic gripper can pick up and identify wide array of objects
• Sticky business: Five adhesives tested for 3D printing
• 3-D printing hydraulically-powered robots, no assembly required

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

UPDATE: New video of a collaborative, cloud-based mapping experiment. Mapping is essential for mobile robots and a cornerstone of many more robotics applications that require a robot to interact with its physical environment. It is widely considered the most difficult perceptual problem in robotics, both from an algorithmic but also from a computational perspective. Mapping essentially requires solving a huge optimization problem over a large amount of images and their extracted features. This requires beefy computers and high-end graphics cards – resulting in power-hungry and expensive robots.

Update

Here is a video of a collaborative mapping experiment using Rapyuta, the open source cloud robotics framework. Instead of just compressing each frame of the stream, we now run a dense visual odometry algorithm to produce keyframes and these keyframes are eventually sent to the cloud for optimization. This technique is a very natural way to compress since keyframes are produced only when the robot has moved a certain distance or angle. The Cloud, in addition to optimization, now also does merging of maps produced by multiple robots. The code is open under the Apache 2.0 license at https://github.com/rapyuta/rapyuta-mapping.

Together with our RoboEarth colleagues Dominique Hunziker, Dhananjay Sathe, Mayank Singh, we have set up an inexpensive, light weight robot so that it can perform full 3D mapping in real-time by offloading heavy computation to the RoboEarth Cloud Engine (nickname: Rapyuta) (see previous Robohub article). In the video below Master student Vlad Usenko shows how robots can sidestep this problem.

How it works
What actually happens here? As you can see from the video and the figure below, the robot is equipped with an RGBD sensor (ASUS Xtion PRO), an ARM-based single board computer (Odriod-U2), and an off-the-shelf wireless dongle. The robot transmits RGB images (QVGA resolution) and depth images (QVGA resolution) at 30 FPS to Rapyuta with a frame by frame compression (JPEG for RGB, and PNG for depth images). The single board computer is only used for compression, communication, and low-level control of the robot – the heavy computation all happens off-board. The optimized map shown at the end of the video is built by manually triggering the optimization process in the cloud after the demo (i.e., not in real time).

The main purpose of this demonstrator is to showcase that today’s wireless technologies allow robots to outsource computationally heavy data processing to the cloud. As shown in the figure below, and as I’ve laid out in an earlier article on Cloud Robotics, bandwidth is a key driver for cloud-based robot services and applications.

Most actual mapping scenarios call for a slightly different split in work-load: You’d want to perform visual odometry locally and only send keyframes to Rapyuta for global optimization. The good news is that we already supports this setup: Rapyuta provides the corresponding software packages to run on ODroid boards along with the necessary cloud computing infrastructure.

What’s next?
This demonstration stops short of actually using the data returned from the cloud: A map is created and available in real time, but not actively used to control the robot. Using the map to control the robot, e.g. to improve exploration, is one logical extension.

Another natural next step would be to use the computational resources in the cloud for localizing in an existing map. Imagine a robot entering a new building and trying to get its bearings. Such localization in an existing map requires performing a search over the entire map – essentially a very large set of images and their features – to find the best match between what the robot is seeing now and a part of the map.

But the cloud can be used to do much more interesting things than improve single robot navigation. Multiple robots can instantaneously share their maps, so applications like multi-robot mapping become much easier. This implies using the cloud as more than a pure computational resource – it becomes a medium to store, share, and collectively improve data. This is interesting for static scenes, but things get even more interesting for dynamic scenes, for example when multiple robots track dynamic objects in the map. With WIRE, some of my RoboEarth colleagues have developed algorithms to do just that. We are currently working on integrating this functionality with Rapyuta and the RoboEarth database and will make it available as a Rapyuta cloud service later this year.

Many of the key advantages of using this Cloud Robotics approach to mapping will only become apparent when maps are brought together with the vast amounts of knowledge required to understand environments. Performing mapping in the cloud not only allows the creation of maps but also the ability to understand them: By bringing computation close to the knowledge required to make sense of all of a robot’s sensor information, Cloud Robotics offers robots a very powerful way to understand the world around them. When working together, cloud computing frameworks like Rapyuta and online knowledge bases like RoboEarth can allow us to make robots that are both smart and inexpensive.

Additional information
For further information, see:

• Technical details and tutorial for this setup (available early July)
• Video of recent live Cloud Robotics Mapping demo at ROSCon 2013
• Cloud Robotics Workshop at IROS 2013 in November, Tokyo
• Homepage of the RoboEarth Cloud Engine (codename “Rapyuta”)
• RoboEarth homepage

Questions?
Feel free to post in the comments below.

Tutorial Structure

The entire tutorial takes about 10 hours to complete and is tailored towards students aged 12 and above.

The project will be presented in six tutorials:

1. In the first tutorial (this one) we will be looking at how the DanceBot works, and will be covering how to read circuit diagrams.
2. In the second tutorial, we will look at how to construct the DanceBot’s power supply.
3. In the third tutorial, we will be building the DanceBot’s audio amplifier.
4. In the fourth tutorial, we will build the DanceBot’s micro controller, digital line converter and H-bridge.
5. In the fifth tutorial, we will assemble the DanceBot’s LEDs.
6. And in the final part, we will look at the mechanical construction, and how to program your DanceBot to groove to your favorite tunes!

But before we get started, let’s take a look at the final product:

http://www.youtube.com/watch?v=XDDs63Z69Tc

Fun, funky and fabulous, but how does it work?

The picture below is a rough sketch of what we’ll be building in these tutorials. You’ll see some common elements found on many robots.

Some of the major components that you’ll find in the robot are:

1. Light Emitting Diodes (LEDs)
2. A speaker
3. Thick gauge wire
4. Electronics
5. A battery pack
6. An MP3-player
7. A base
8. Two wheels
9. Two motors

The key to how all of this works is the music that is played by your MP3-player. There are generally two channels (i.e. left-channel and right-channel) for music that you play on your MP3-player. These channels send music to your left and right ear, respectively. In the third part of the tutorial series, using the provided software, you will program dance moves into your favorite song by replacing the right-channel of music with binary-encoded dance moves! When your MP3-player is plugged in and you play your encoded song, the left-channel goes directly to an audio amplifier and plays the music through the speaker, while the right-channel goes to a small computer, which interprets the digital information as motor and light commands.

The electronic roadmap

A schematic is a drawing that shows us how electronic components connect together and where they are placed on the printed circuit board (PCB, the picture below). If you, like most people, don’t have the ability to print a circuit board, don’t worry, prototyping boards will work just fine!

Before we start assembling the electronics, it’s important to know how to read the DanceBot’s schematic. You can download the schematics for this project here. Below you will find a list of common elements, which you can identify on the schematic and on the PCB.

Component Type	Measurement Unit	Symbol	Picture
Resistor	Ohm (Ω)
Capacitor (polarized)	Farad (F)
Capacitor (not polarized)	Farad (F)
Light Emitting Diode (LED)	–

Now take a look at the schematic. Notice that beside each symbol in the schematic is a name (black) and a value (blue). The Name helps to identify the location of a part on the PCB (the picture above), while the value identifies the value of the part, as described in the table above.

In addition to name and value, some parts have a polarity. Be careful! This means you need to put them in the right way, otherwise things won’t work and could get damaged! Diodes have a polarity, and in this project some capacitors have a polarity too. In the next tutorial, where you will assemble the DanceBot’s electronics, we will warn you of any components with a polarity.

That’s it for now

In the next two tutorials, we will put together the DanceBot’s electronics, and then build its body and program some funky moves. If you would like to follow along in future tutorials, you’ll need to organize a few things. You’ll need to:

• Download the components list and buy the (few) components for the MP3 DanceBot Kit
• Make the mechanical base template
• Organise a soldering iron, solder, a hot glue gun, a multimeter and some side-cutters

If you don’t know where to start with these steps, a friend with electronics experience will definitely be able to help!

Once you have organized the above, take some time to learn how to solder and learn how to use a multimeter to check connections.

http://www.youtube.com/watch?v=jLt_xbyYq8Y

Stay tuned for the second tutorial!

My interest in mechanical things came about from the age of five. I distinctly remember carrying around a little sketchbook with a pencil everywhere I went. Whenever I had an idea for something, I would draw a picture in a page of the book, maybe writing a word or two on what it was. Most of the time these ideas were Rube Goldberg machines designed to do mundane tasks like watering plants or making hot drinks. As I got older and played around more with LEGO Technic, the complexity of these machines and the information I’d write about them increased. By the age of eight I was starting to become more interested in electronics. I loved designing mechanical machines, and with electronics I could make the machines I was designing much more complicated. I began to pull apart my toys due to an insatiable curiosity to see how they worked. My knowledge, of course, was very basic – the only electronic parts I knew of were the ones with a visual function, such as a DC motor or an LED. Even the sketchbooks were getting an electronic overhaul; I remember a few of them had the LCD clock faces on the front cover that I pulled out of my wrist watches.

By the age of ten, the designs I was drawing in my sketchbook were more product designs. I designed phones, watches, cars, computers and even game consoles. The designs had all the key components diagrammed and labelled with labels like ‘CD reader’ and ‘Big board’ (Motherboard). By this stage the designing process became a key interest as I spent more time trying to make the designs look good so that I could show others my work.

I think the step between machines and product designs to robots was due to two key things that were happening during that time. The Star Wars episodes 1-3 were released during my early childhood and, when I was old enough, we hired the DVDs. I was instantly hooked and loved the idea of C3PO. The fact that Anakin made C3PO in order to help his mother specifically seemed to target me, it was like a sign that I needed to do the same.

The other reason I became interested in robotics was actually to do with my cousin. Every Thursday night I would go to his house and play computer games with him. One night I went there and he had this really cool game called Robot Arena 2, in which you design battle bots. The game let you customize everything from the body shape to the internal components and the weapons you used. I spent hours playing this game and absolutely loved it.

I borrowed all the books in my school library that had anything to do with robots (there weren’t many) and started to design robots.

At the age of fourteen my parents bought me the Lego Mindstorms NXT. The NXT was a Lego kit that let you build and program your own robot. I played with it for around two years and had great fun with it, but soon my hobby was going to take a turn for the worse.

I was in high school now and things were getting tough. I was starting to get really stressed at school and began to play video games as a release. Video games quickly took over my life. I would get home from school at 4PM and jump onto my computer to play. I would only get off for tea around 7PM and then get straight back on until I had to go to bed. It was a poisonous cycle and I regret those years of my life as I felt like I achieved nothing. My video game addiction meant I didn’t do any homework and spent no time revising what we covered in class. My grades began to slip and my gaming habit only became worse.

The cycle was only broken when it came to crunch time, Year 12, the final year of high school. I realised what I had gotten myself into and was determined to make it stop. I put my head down and worked like I never had before. My grades picked up dramatically in all subjects but Math. I studied every night and even finished assignments early. Sadly it was too late for Math, I was never good at it when I was young and the period of not doing homework absolutely destroyed any understanding I had on the subject.

During the mid-year break I wanted a new hobby, a hobby that would not only be something challenging like computer games, but also something that would actually benefit me. I remembered how much fun I had designing and making robots and decided it was time to go back to that.

I went online to research other ways I could make robots other than using Lego, that was when I heard of this thing called an Arduino. I didn’t really understand what it was but I bought one called an Arduino Uno. When it arrived in the post I looked up tutorials to see how I could get it to do things.

July 18 2012 I uploaded my first video to YouTube on Arduino. It wasn’t my first creation, but it was the first one that I had recorded. I recorded it on my phone and just uploaded it straight to YouTube. I had uploaded previous videos before on YouTube and thought it would be cool to show off what I had made.

From that point on I recorded all of my creations and uploaded them to YouTube, each time writing a little about them on my blog.

First I covered all the basics, such as making sounds, lighting LEDs in patterns and detecting light changes. When I had used up all the components that came in my starter kit I researched ways to make things move with the Arduino. I found these things called servos and read they were easy to use so I went on Ebay and bought a few of them cheap from Hong Kong. Once I had worked out how to use them I made my first little bot. It was nothing complex and didn’t even have any autonomous capabilities. The robot simply moved the servos to certain positions to make the robot hop like a frog. Money was scarce so I made things out of whatever I could. The body was made of cardboard and the legs were the servo arms with heat shrink over the top to add grip.

I had experience with programming before and I was pretty good at computer software, so once I understood how I could make robots with the Arduino, I began to make them smarter and more complicated.

I became interested in program in particular called GlovePIE. GlovePIE is a program that allows you to program gamepads to act as a keyboard and mouse. I wrote a code that allowed me to have full control of my computer by simply talking, I called it JARVIS.

After programming JARVIS I realised that I could use voice recognition to control the Arduino. To test this theory I made a humanoid robot that was able to express emotions and communicate.

I also learned how to use infrared commands and turned my Arduino into a voice controlled TV remote.

It had only been two months but I absolutely loved making things with Arduino and couldn’t stop. Every time I wanted to learn something new I researched for hours on how to do it and watched so many YouTube videos I used up our internet allowance. I went from controlling a remote control car with my iPhone, to making a webcam that tracked my every movement.

My creations were becoming more and more complicated and the views my videos were getting was increasing. I was getting asked so many questions a day I decided I would start doing tutorials. I was getting more comfortable with YouTube as the response to my creations was positive. I kept making tutorials covering all the basic things you needed to make robots with Arduino.

My robots were starting to get very complicated and had various autonomous capabilities. My friend from school had a big box full of old robot kit parts that he had collected as part of a magazine series, he knew I was interested in robotics so he kindly gave me all the parts. I salvaged everything and turned the brain of the bot into an Arduino.

I began to understand more about the electronics inside robots and my knack for software proved invaluable in my first big robot project.

I had five tutorials released by this stage and they were becoming very popular, with thousands of views. My old physics teacher, who I am good friends with, suggested I come into school and teach the students about robotics. I loved the idea and since then I have been volunteering every Friday during their lunchtime to teach the students about robotics and programming.

I kept making robots and developing new skills. My videos were getting a lot of views now and I had just over five hundred subscribers. I was now recording everything I did, whether it was about robots, making things to help with videos or just me going out to conventions.

I thought it was time now to start making videos focused purely on making robots. I made several videos on the matter and three Instructables on three different bots, Ben, James and Yogy. I realised now that people were really starting to like my creations as my bots had started to win me prizes.

Yogy the yoghurt-tub robot!

After spending a semester at the University of South Australia, studying a bachelor of Mechatronic Engineering, I decided to take a one year deferment. I choose to do this for several reasons. The first major reason was the job prospects. We’d often have people from the field come into lectures and talk with us about what they do. The one thing that I hated about what they all said was that none of them were hands on, they all sat behind a desk and worked for the ‘big man’. As I spent more time volunteering at the school I realized that perhaps teaching is the job for me. I will always love making robots, but sometimes I feel like sitting behind the desk just isn’t for me. Teaching seems to bring my love for robots and the enjoyment I get out of teaching together.

I now have over one thousand subscribers on YouTube and am starting to make of money off of it. Recently, a publishing agent contacted me asking whether I would be interested in producing a video course for them on the subject of ‘Making Arduino Powered Robots’. I agreed to their offer and am working with them now to produce the course.­

In the next few weeks I will make the transition from not knowing what to do with my life, to turning my forever interest in robotics into a job. I am going to be an author and also be helping a robotics kit creator in writing a teaching curriculum and helping with instructional videos for his kits. I feel like this is the start of a lifetime journey and I can’t wait to see where robotics takes me.

I don’t know where I am going, but I feel like I am surfing the wave of opportunities. I hope one day to be producing my own kits or perhaps working with the likes of MAKE promoting robotics for kids.

I would also like to mention one last thing. Without the people on LetsMakeRobots.com I probably wouldn’t be where I am now. They are the people who have stood behind me, giving me support and help when I needed it, and have been my slingshot into having a robotics-based career. If you are interested in making your first robot, or have been making robots for a while, go to their site and join. LetsMakeRobots has a great community for both beginners and veterans alike, they have a step by step instructional on how to make your first bot and you can even show off your robotics projects.

I hope you can draw similarities between my story and yours. Remember, stories are ongoing and the more effort you put in, the better your story will be.

We take you now to sunny, southern California, where a small group of enthusiasts has constructed a very realistic, Arduino-based replica of Pixar’s WALL-E, entirely from custom-fabricated parts.

The beloved Wall-E robot was just computer generated graphics in the Pixar movie, but fans have spent years trying to bring him to life. We visit Mike McMaster’s workshop to see his incredible life-size Wall-E, a remote controlled robot that lives among an R2-D2 droid and other pets on Mike’s orange farm.

Up to 10,000 young geeks from across Europe will camp out in tents and sleeping bags for five days of intense 24/7 hacking, networking and elbow-rubbing with experts from some of the world’s most influential science and tech communities. The main goals of this hands-on event are to debate the biggest issues in digital technologies, and to inspire a new generation of European techpreneurs. Campus Party runs September 2 -7 at the O2 in London.

A massive digital skills marketplace will sit at the heart of the festival, surrounded by dedicated stages for robotics, start-ups, sci-tech, Big Data, social networking, and gaming. Lectures, workshops, competitions and hackathons take place simultaneously.

Two-person tents are set up in a secure area just a few minutes walk away from the tech stages, so that “Campuseros” who want a 24/7 experience can hack to their heart’s content.

Ronan Dunne, CEO of Telefonica (the main sponsor of this year’s event) says: “Since 1997 tech fans across Europe and Latin America have flocked to Campus Party in the thousands to innovate, learn and engage with some of the biggest names in technology. This year the marketplace will feature a huge range of free support services, including advice on how to get your foot on the career ladder or how a business can succeed on Facebook. The event will help people hone and develop their digital skills whilst giving businesses of all sizes the opportunity to find the talent they need to grow.”

The festival will feature some of the world’s most inspirational science and technology communicators. Confirmed speakers include:

•  Alex ‘Sandy’ Pentland, foremost authority on organizational engineering, mobile information systems, and computational social science, and founder and director of the Human Dynamics group and the Media Lab Entrepreneurship Program at MIT.
•  Jon “maddog” Hall, pioneer in the world of commercial computing and a globally recognised authority on free and open source software, and 2006 winner of the UK Linux and Open Source Lifetime Recognition Award for his services to the open source community. His latest ambition is Project Cauã.
• Nolan Bushnell, founder of Atari Inc. and father of the modern video game industry. Bushnell created the world’s first technology incubator, the world’s first touchscreen menu ordering and entertainment system, the world’s first digital navigation system and the world’s first online ordering system.

Find out more information or pre-register at http://www.campus-party.eu/. Members of the Robohub community receive a 20% discount off Campus Party tickets. Businesses and educational organisations keen to be at the Digital Skills Marketplace can email campusparty@telefonica.com for more information.

[Photo credit: Christiano Sant’Anna]

This is the third tutorial in the Up and flying with the AR.Drone and ROS series.

In this tutorial we will:

1. Learn about the AR.Drone’s state feedback (and how it is handled by ROS)
2. Learn about the AR.Drone’s tag detection
3. Program our first ROS nodes: A subscriber and a publisher

In the previous tutorials we:

1. Installed ROS, the AR.Drone driver and AR.Drone keyboard controller then flew the AR.Drone using the provided keyboard controller (link)
2. Learned about ROS communication, setup and then flew the AR.Drone using a joystick (link)

In the next tutorials, we will:

• Write a controller to enable tag following
• Write a controller for the drone which allows us to control drone velocity, rather than body angle

What you’ll need for the tutorial

1. An AR.Drone!
2. A computer or laptop with WiFi
3. Linux, either the ARDroneUbuntu virtual machine, or a custom installation
4. Either the keyboard or joystick controller from the previous tutorials

I also assume that you’ve completed either the keyboard or joystick tutorial, which detail how to setup the software, connect to the drone, etc, and provide a method of controlling the AR.Drone from your computer.

Updating

If you have not completed the joystick tutorial:

Please go back and update your installation according to the instructions listed there.

If you downloaded the ARDroneUbuntu virtual machine:

To update your installation for this tutorial, click the “Update Tutorials” button in the ARDroneUbuntu Virtual Machine

If you use a custom installation:

At the terminal, enter the following:

$ roscd ardrone_autonomy
$ git pull
$ roscd ardrone_tutorials
$ git pull
$ roscd
$ rosmake ardrone_tutorials

AR.Drone State Feedback – A First Look

Preparation:

1. Find the Orange-[Green|Yellow|Blue]-Orange tag that came with the AR.Drone.
2. In your ARDroneUbunutu virtual machine (or custom installation), open a new terminal window and type:

   $ roscd ardrone_tutorials/launch

3. If you intend to fly with the keyboard, type:

   $ geany keyboard_controller_with_tags.launch

   Otherwise, if you intend to fly with the joystick:

   $ geany joystick_controller.launch

4. Now find the line:

   <param name="enemy_colors" value="3" />

5. And change the value to “1”, “2”, or “3”, depending on whether your tags are Orange-Green-Orange, Orange-Yellow-Orange or Orange-Blue-Orange respectively.

Tag Detection and Feedback Data

1. Run the system (but don’t takeoff with the drone!) using either the keyboard_controller:

   $ roslaunch ardrone_tutorials keyboard_controller_with_tags.launch

   or the joystick_controller:

   $ roslaunch ardrone_tutorials joystick_controller.launch

2. Open a new terminal window and type:

   $ rostopic list

   • This list shows all the active ROS “topics” used for communication to and from the AR.Drone
3. In the same terminal window, type:

   $ rostopic echo /ardrone/navdata

   • This echoes the AR.Drone’s feedback data into the terminal window.
   • Note that many items will be zero because the AR.Drone is not yet flying
4. Move the Orange-Color-Orange tag in front of the AR.Drone’s front camera. It should be detected, and the console output (from the previous step), should update to reflect this detection.
   • Note the tags_xc and tags_yc values. These represent the x/y location of the tag within the image.
   • The video display window will show a dot at these coordinates, and the distance measurement above the dot
5. Now place the tag somewhere that can be seen by the flying drone. After this is done, take off with the drone.
6. Watch the console output. What happens if you fly forwards? sideways? What variable changes if you rotate the drone?

Plotting the AR.Drone’s State

1. A more intuitive way of representing this feedback data is using a plot. PressCtrl-C to terminate the echo process from the last stage, and then type:

   $ rxplot /ardrone/navdata/rotX,/ardrone/navdata/rotY

   • This launches a plotting window showing the AR.Drone’s estimated roll (rotX) and pitch (rotY).
   • Similar plots could, for example, be generated for yaw (rotZ), x and y velocity (vx and vy), altitude (altd), etc.
2. Similarly to displaying the AR.Drone’s feedback data, we can use the plotting method to investigate how we are controlling the drone:

   $ rxplot /cmd_vel/linear/y,/cmd_vel/linear/x

   • This plots the roll (linear/y) and pitch (linear/x) commands.
   • Note that these are scaled with a value of $\pm1$ being the maximum commandable angle (set by the euler_angle_max in the launch file).
   • Note also that the direction of AR.Drone movement is commanded (ie. move along the x-axis = /cmd_vel/linear/x = commanded pitch), whereas the AR.Drone’s feedback data is named with reference to axis of rotation (ie. tilt around y-axis = /ardrone/navdata/rotY = feedback pitch).

Logging Data

1. To log data, we can use the inbuilt ROS command rosbag:

   $ rosbag record -O ARDroneFlight.bag /ardrone/navdata /cmd_vel

   • This will record the topics /ardrone/navdata and /cmd_vel to a “bag-file”, stamped with the current date and time.
   • Leave this process running for a few seconds, fly the AR.Drone around, and when finished press Ctrl-C to terminate.
   • Recording all topics is possible with rosbag record -a, however this also records the AR.Drone’s HD video stream, thus bag-files will quickly become very large
   • The advantage of using rosbag to log data, is that bag-files can be replayed at a later time, and interact very nicely with the existing ROS infrastructure.
2. Land the AR.Drone and close any windows / terminate any processes relating to the controller. Now, into the terminal window that ran the AR.Drone controller, type:

   $ roscore

   Now open a new terminal window and type:

   $ rxplot /cmd_vel/linear/y,/cmd_vel/linear/x

   Now finally (ignoring the warning from the previous step), switch back to the terminal window that you used to run the rosbag command and type:

   $ rosbag play ARDroneFlight.bag

   Does the flight look familiar? It should! This is the flight you recorded in the last step! Use Ctrl-C to terminate the above processes.

Custom Data Handling

In the last section, we investigated the AR.Drone’s feedback data by using ROS’s inbuilt plotting (rxplot) and logging (rosbag) commands. In this section, you will learn how we get this feedback data into your own application by writing a custom ROS Node. This node will be used to compare the commanded roll and pitch angles with the AR.Drone’s roll and pitch estimates, thus laying the foundations for the next tutorial, where we will use this feedback data to programmatically control the AR.Drone.

Messages, Topics and Nodes

In ROS terminology, a program is known as a Node. To interact with other nodes (for example the AR.Drone driver), each node will have some subscribers, which listen for information messages on a specific topic (communications channel); and some publishers, which send messages to a specific topic.

These messages can be anything. An example that we saw in the previous section was the Navdata message, which is sent over the /ardrone/navdata topic and contains feedback data from the AR.Drone.

We will now investigate this message a little further. Open up a new terminal window and type

rosmsg show ardrone_autonomy/Navdata

You should be presented with the following output:

$ rosmsg show ardrone_autonomy/Navdata
std_msgs/Header header
  uint32 seq
  time stamp
  string frame_id
float32 batteryPercent
uint32 state
int32 magX
int32 magY
int32 magZ
int32 pressure
int32 temp
float32 wind_speed
float32 wind_angle
float32 wind_comp_angle
float32 rotX
float32 rotY
float32 rotZ
int32 altd
float32 vx
float32 vy
float32 vz
float32 ax
float32 ay
float32 az
uint32 tags_count
uint32[] tags_type
uint32[] tags_xc
uint32[] tags_yc
uint32[] tags_width
uint32[] tags_height
float32[] tags_orientation
float32[] tags_distance
float32 tm

This listing shows us the various members of the Navdata message, including their datatypes. Note that the Navdata message has a header, which includes the time the message is sent.

Minimum Viable Subscriber

Open up a new terminal window and type:

$ roscd ardrone_tutorials
$ geany src/minimum_viable_subscriber.py

Up will pop a geany window, ready for your to edit the new file. Enter the following into the new geany window, then save and close the file:

import roslib
import rospy

roslib.load_manifest('ardrone_tutorials')

from ardrone_autonomy.msg import Navdata

def ReceiveData(data):
  print '[{0:.3f}] Pitch: {1:.3f}'.format(data.header.stamp.to_sec(), data.rotY)

rospy.init_node('minimum_viable_subscriber')
sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)

while not rospy.is_shutdown():
  pass

We will now use the previously recorded bag-file to test the node that we’ve just written.

1. In one terminal window, enter:

   $ roscore

   Which begins the roscore process, which coordinates the interactions between nodes.

2. Now in a new terminal window type:

   $ roscd ardrone_tutorials
   $ python src/minimum_viable_subscriber.py

   This will launch the node that we’ve just programmed.

3. Finally, in a third terminal window, begin the bag-file playback with:

   $ rosbag play ARDroneFlight.bag

   • Note you must be in the same directory as the bag-file for the above command to work
4. If you switch back to the terminal window in which our new node is running, you should see a stream of time-stamped printouts with the drone’s pitch, for example:

   [1353596639.491] Pitch: 4.375
   [1353596639.511] Pitch: 4.128
   [1353596639.531] Pitch: 3.750
   [1353596639.552] Pitch: 3.304
   [1353596639.571] Pitch: 3.063

Minimum Viable Subscriber: A Closer Look

If you made it through the previous section, congratulations, you have successfully written and tested a ROS node, which listens for Navdata measurements from the AR.Drone (or equivalently: a recorded flight) and outputs a time-stamped pitch measurement. But how does it all work?

Lets take a look at the source-code from the previous section, line by line:

import roslib
import rospy

With these two lines, we import the standard ROS packages, required for every ROS node.

roslib.load_manifest('ardrone_tutorials')

Once the required packages are imported, load the manifest file. This file details the dependencies of the current project (for example, that ardrone_tutorials is dependent on the AR.Drone driver, ardrone_autonomy), which through the above command are loaded into the path.

from ardrone_autonomy.msg import Navdata

Once the project dependencies have been loaded, we can import those which are needed by the current node. In the above case, we load the Navdata message definition from the ardrone_autonomy package.

def ReceiveData(data):
  print '[{0:.3f}] Pitch: {1:.3f}'.format(data.header.stamp.to_sec(), data.rotY)

Here we define a callback function to handle the Navdata messages. This function prints a formatted string containing the message’s timestamp and the AR.Drone’s pitch angle. This callback function will be used later in the program.

rospy.init_node('minimum_viable_subscriber')

Before issuing any commands to the rospy package, we need to initialize a ROS node.

sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)

We can now subscribe the node to various topics. Above we subscribe the node to the /ardrone/navdata topic, which carries Navdata messages. Received messages should then be handled by the previously-defined ReceiveData function.

while not rospy.is_shutdown():
  pass

Because our program is relying on callbacks for data processing, we need to stop the program from terminating. The above defines a loop that runs until the node is shut down (for example with Ctrl-C). In most situations the above loop would perform some sort of processing, however in our example, we simply pass (do nothing).

Publishing Data

In this section we will introduce the concept of publishing data to a topic (the opposite of subscribing). We will be using this more extensively in the next tutorial, where we will be programmatically controlling the AR.Drone by publishing messages to the /cmd_vel topic.

Minimum Viable Publisher

The goal of this section is to create a minimum_viable_publisher.py file such that Navdata messages are averaged over a second and the average pitch published to the (new) /ardrone/average_pitch topic. Modify the file as follows:

import roslib
import rospy

roslib.load_manifest('ardrone_tutorials')

from ardrone_autonomy.msg import Navdata
from std_msgs.msg import String

messages = []

def ReceiveData(data):
  messages.append(data)

rospy.init_node('minimum_viable_publisher')
sub_Navdata = rospy.Subscriber('/ardrone/navdata', Navdata, ReceiveData)
pub_Average = rospy.Publisher('/ardrone/average_pitch', String)

r = rospy.Rate(1)
while not rospy.is_shutdown():
  if len(messages)>0:
    avg = sum([m.rotY for m in messages])/len(messages)
    messages = []

    avgmsg = String()
    avgmsg.data = 'Average Pitch: {0:.3f}'.format(avg)
    pub_Average.publish(avgmsg)
  r.sleep()

You can run the program using the following series of commands:

1. In one terminal, run:

   $ roscore

2. In the next terminal, run the new program

   $ roscd ardrone_tutorials
   $ python src/minimum_viable_publisher.py

3. We now echo the published data to the topic we’ve created:

   $ rostopic echo /ardrone/average_pitch

4. And finally, like we’ve done twice before, we begin the playback of our recorded flight

   $ rosbag play ARDroneFlight.bag

5. Switching back to the terminal in which we ranrostopic echo, you should see a new message with the average pitch appearing once a second:

   data: Average Pitch: 0.649
   ---
   data: Average Pitch: 1.078
   ---
   data: Average Pitch: 3.781
   ---
   data: Average Pitch: 3.285
   ---

Minimum Viable Publisher: A Closer Look

Comparing minimum_viable_publisher.py to minimum_viable_subscriber.py from the last section, we can see many similarities:

from ardrone_autonomy.msg import Navdata
from std_msgs.msg import String

We additionally import the String message, required for our new topic.

messages = []
def ReceiveData(data):
  messages.append(data)

We have updated the navdata callback function to save the messages in a queue, rather than process them straight away.

Note: because we’re accessing the messages list in both a callback function and the main loop, we should be using Python’s threading.Lock to control access to the list and prevent race conditions. See ardrone_tutorials/src/drone_video_display.py for an example.

pub_Average = rospy.Publisher('/ardrone/average_pitch', String)

We create a new publisher, responsible for publishing messages of the String type to the /ardrone/average_pitch topic. Note that we do not need to explicitly initialize this topic, it will be initialized automatically on first usage.

The biggest change is in the program’s main loop, which I will now cover in a little more detail:

r = rospy.Rate(1)

Firstly we define a Rate object. We use this to control the frequency of the main processing loop. In our case, we want to print a message every second, thus we want a frequency of $1 \text{Hz}$.

while not rospy.is_shutdown():
  if len(messages)>0:
    avg = sum([m.rotY for m in messages])/len(messages)
    messages = []

    avgmsg = String()
    avgmsg.data = 'Average Pitch: {0:.3f}'.format(avg)
    pub_Average.publish(avgmsg)

If messages have been received, we calculate the average pitch using a python list comprehension to extract the pitch (.rotY) from every Navdata message. We then empty the messages queue.

We then construct a new String message and set its data property, before publishing it to the topic we previously created.

r.sleep()

Finally, we tell the rate object to sleep. This command will calculate how long it needs to sleep, such that the desired loop frequency is achieved.

Conclusions

I hope you’ve enjoyed working through this tutorial and learning about data flow in the ROS environment. In the next tutorial, which I will release in a month’s time, we will use what we have learned today to programmatically control the AR.Drone, while at the same time learning about PID control.

If you have any questions about the tutorial, please leave me a comment below and I will get back to you as soon as possible!

You can see all the tutorials in here: Up and flying with the AR.Drone and ROS.

If you liked this article, you may also be interested in:

• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Classroom robotics: Motivating independent learning and discovery

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In particular, Anderson promoted one idea that deserves the attention of the drone industry: that by operating in a legal “grey area” (read: deliberately violating regulations that are perceived as unreasonable and unenforceable) drone enthusiasts will force regulatory movement, and this is a good thing. Anderson gave two examples of circumstances that challenge current regulations: Lego Mindstorms being classified by regulators as a cruise missile guidance system (and hence violating export controls), and flying drones in contravention to the FAA regulations governing the use of unmanned aircraft.

Technologists in general (and Silicon Valley in particular) take a dim view of regulations promulgated under the old order (e.g. Lyft, Uber, AirBnB), and it’s true that some aspects of the FAA regulations are out-of-date and no longer reasonable. But while I’m not completely outraged when technologists upend the status quo to make something new (and improved), this attitude works better in industries that don’t involve fast-flying heavy objects.

Let’s remember that the FAA’s primary goal is to keep people from getting killed by aircraft. When it comes to aircraft regulation – manned or unmanned – people on the ground are the main beneficiaries. We are not talking about regulating a relatively low-risk activity (like renting your room out with AirBnB); we are talking about regulating heavy objects hurtling over the heads of people who haven’t consented to be part of the experiment.

Our society rightly asks the government to ensure that activities that impose risk on others — especially those who did not consent — be minimized. We need to update our regulations, but aircraft operators need to respect the letter and spirit of the law as it stands.  What standards do we have if not the law? If we follow the path of breaking all the rules, someone is going to accidentally kill a sympathetic bystander. Beyond the personal tragedy this will create, such an accident will set back our industry and the benefits we can provide to society by a decade.

In Afghanistan, one of my planes almost smacked into a helicopter, but it was the helicopter – not the drone –  that had come, without clearance or radio calls, into an active artillery firing ROZ (restricted operating zone: this is an airspace control measure that makes sure that aircraft don’t run into artillery fire).  Similarly, the first full-sized drone and manned aircraft collision involved a C-130 violating airspace control measures around an airfield.

Pilots are not infallible and often break the rules. The best drone operators have a different safety culture: one where there is proper approval for everything, because every move will be recorded and second guessed. Military drone safety culture is a great example of this approach. I hope this culture will permeate the civilian unmanned aircraft community as well.

Although breaking the rules might move us toward our unmanned enabled future a little bit faster, this is an incredibly dangerous path for our industry and our bystanders.  The closing thought of VLAB Drones was that unmanned aircraft will eventually make the airspace safer for all users.  The panel wondered if this was hyperbole, but it is inevitable when drones have a strong safety culture. Conversely, as long as we are the irresponsible jerks of the air, safety conscious regulators like the FAA will be unsupportive of us flying.  When we, as a civilian unmanned aircraft industry, can be counted on as strong safety partner, and when people in general and commercial aviation are learning safety lessons from us, there will be no more foot dragging. We will get our airspace and the drone revolution will finally be here.

In the meantime: seriously, don’t do anything that could kill anyone, please. Especially if it is illegal. You will ruin it for the rest of us.

“Moore’s Law has never moved faster than is moving inside the phone you’ve already got in your pocket. The pace of development and the price performance curve is moving faster in smartphones than it’s ever moved in history and we’re taking advantage by drafting off this momentum and by employing military-grade technologies at toy prices,” said Chris Anderson at last weekend’s Engadget Expand event at Fort Mason, San Francisco.

The military has deployed 7,500 aerial drones; Chris Anderson’s company, DIY Drones, has got 40,000. Every sort of copter and plane, as shown on the diagram below, is included in the universal autopilot chipset that DIY Drones sells (for $129). And this chip also includes features like geofencing, follow me, and fly by wire.

Anderson’s fast-paced presentation is well worth watching particularly when he discusses how drones can be used for precision agriculture right now, today:

Chris was just part of a really interesting 3-person panel where the other two presenters were Marc Raibert from Boston Dynamics and Steven Cousins from Willow Garage. Marc Raibert is CEO of Boston Dynamics (of Big Dog and Cheeta fame) and described how and why Boston Dynamics builds robots designed to work in rough terrain. He emphasized what Chris said earlier, that military grade technology was moving to the robotics industry at toy prices subsidized by DARPA and smartphone buyers.

Amongst other interesting things Cousins discussed, he also answered the question about what is going to happen at Willow Garage now that the funding has stopped.

This video, by MotherboardTV, was posted on YouTube in early December, but, at just over 50,000 views, it hasn’t yet reached the audience it deserves.

Not knowing much about drones to begin with, the MotherboardTV team started out by talking with P.W. Singer, author of “Wired for War”. From there, they flew to Amman, Jordan, to attend a military trade show, where they met Chris Barter, Scout Program Manager at Datron. Accepting his invitation to vists the Datron campus in San Diego, they got an opportunity to fly the Scout for themselves. (The Scout is produced by Canadian firm Aeryon Labs and marketed by Datron.)

While in Southern California, they also stopped by 3D Robotics for a chat with Chris Anderson and a couple of their engineers, filling out the picture with the DIY perspective.

All in all, it’s very well done. You may find yourself subscribing to MotherboardTV’s YouTube channel.

View on YouTube

“Electronics supplier SparkFun designs dozens of products a year and they haven’t patented a single one. It’s worked out pretty well so far.”

See full article on wired.com

This is the second tutorial in the Up and flying with the AR.Drone and ROS series.

In this tutorial we will:

1. Talk about the ROS communication hierarchy
2. Setup a joystick to work with ROS
3. Fly the AR.Drone with a joystick

In the previous tutorial we:

1. Installed ROS, the AR.Drone driver and AR.Drone keyboard controller then flew the AR.Drone using the provided keyboard controller (link)

In the next tutorials, we will:

• Look more closely at the data sent back from the drone and play with the on-board tag detection (link)
• Write a controller for the drone to enable tag following
• Write a controller for the drone which allows us to control drone velocity, rather than body angle

What you’ll need for the tutorial

To complete this tutorial you will need:

1. An AR.Drone!
2. A computer or laptop with WiFi
3. Linux, either the ARDroneUbuntu virtual machine, or a custom installation
4. A  (linux compatible) USB joystick or control pad

I also assume that you’ve completed the previous tutorial, which details how to setup the software, connect to the drone, etc.

Updating

It has been three weeks since the launch of the first tutorial and I am very happy with the response so far. The tutorial has been viewed over 720 times and ARDroneUbuntu downloaded over 50 times. I was also given the honor of partaking in RobotGarden’s very first Quadrocoptor Workshop, where we worked through the tutorial (and ironed out the bugs) together.

This response has taught me a few things about how to structure a tutorial series, and how to release a virtual machine. Based on what I have learned, I will be restructuring the source-code for the following tutorials. I will also be updating the virtual machine to reflect these changes.

When writing the first tutorial, I thought it would be better to have a separate repository for each tutorial to keep everything atomic and isolated to ensure the addition of a new tutorial would not affect anything in the past – However having learned from the video-hangout with RobotGarden, I realize this could cause an update nightmare down the track. For this reason, I have decided to operate from a single repository and will maintain a stable branch for each tutorial, with the master branch always synchronized with the stable version of the latest tutorial. The source-code for each tutorial will also contain all source-code required to run the previous tutorials.

What this means for you:

• If you downloaded the ARDroneUbuntu virtual machine before the release of this tutorial, you will need to click the “Update Tutorials” button twice. After this, your virtual machine will be updated to reflect the new changes.
• If you installed the tutorials manually, you will need to run the following at the terminal:

  # roscd
  # rm -rf ardrone_tutorials_getting_started
  # git clone https://github.com/mikehamer/ardrone_tutorials

• If you have not yet downloaded the VM, no problems! I have released an up-to-date version that you can download from here

Sorry for any hassle caused by the above, I hope this new structure will allow me to more easily deliver the next tutorials!

So without further ado, lets check out some data!

1. Connect to the AR.Drone as detailed in the first tutorial (remembering to disconnect other network adaptors if they use the 192.168.1.x address range)
2. From within your Linux installation, launch two terminal windows (ARDroneUbuntu users, refer to the first tutorial if you’re not sure what I mean)
3. In the first window, launch the KeyboardController from the first tutorial, using the command:

   $ roslaunch ardrone_tutorials keyboard_controller.launch

4. Wait for everything to load and the video display to become active, then in the second terminal window enter the command:

   $ rostopic list

   This command lists all active “topics”, the communication channels through which ROS programs (called “nodes”) communicate.

5. To view the currently running ROS Nodes, type:

   $ rxgraph

   The window that appears will show a network of nodes, and how they are connected through the topics that we saw in the previous step.

6. When you’re ready, close this window. We’re now going to have a look at the messages that are being sent between nodes. At the terminal, type:

   $ rostopic echo /ardrone/navdata

   The navdata message is published by the AR.Drone driver at a rate of 50Hz.

7. Press Ctrl-C to stop echoing the navdata message then switch to the other terminal window and press Ctrl-C again, to stop the roscore program

So what are we setting up today?

After that whirlwind tour of the ROS communication hierarchy, lets take a look at what we’ll be building today:

As shown above, today we’ll be setting up the joy_node (a standard ROS package) to listen for input from a joystick or control-pad. This input will be converted into a ROS message, then sent to our controller over the joy topic.

Our controller subscribes to the joy topic and reacts to the received messages by translating joystick movement into drone movement commands, which are then published to the cmd_vel topic; and button presses to takeoff, land, or emergency commands, which are each published to a respective channel.

The ardrone_driver receives these various commands and then sends them via the wireless network to the physical drone, which in turn sends back a video stream. This video stream is published by the ardrone_driver onto the /ardrone/image_raw topic, from where it is received and subsequently displayed by our controller.

Setting up a joystick

Firstly, we need to connect the joystick and set it up to work with ROS.

1. Open a terminal window (or reuse one from before), then type:

   $ ls /dev/input

2. Take note of the jsX devices (where X is a number). These are not your joystick.
3. Now connect your joystick to the computer and again type:

   $ ls /dev/input

4. If you see a new jsX device appear, congratulations, you’ve successfully connected the joystick, take note of which device this is, then progress to the next section of the tutorial
5. If no new jsX device appears when you connect the joystick, you’re most likely using a virtual machine and may need to add the USB device manually.
6. If you are using the ARDroneUbuntu virtual machine, open the Machine > Settings window, then the Ports > USB tab. With the joystick plugged in, add a new USB device. Your joystick should appear in the dropdown list.
7. Click OK, then unplug the joystick from the computer
8. Repeat steps 3 and 4, plugging the joystick into a different USB port if it is not detected in step 4.

Testing the joystick

Now that we have the joystick installed, let’s check to make sure everything is working. Type the following at a terminal, substituting jsX for the joystick device you previously identified:

$ jstest /dev/input/jsX

If you are greeted with a “jstest: Permission denied” error, type:

$ sudo chmod a+r /dev/input/jsX

Which enables all users to read the joystick device (ARDroneUbuntu users, the password is: ardrone). After running jstest, moving the joystick should cause the numbers on the screen to update, if not, try running jstest for the other jsX devices. Congratulations, your joystick device is working! Press Ctrl-C to exit.

Using the joystick with ROS

Now we’re ready to integrate the joystick device with the ROS ecosystem using ROS’s “Joy” package.

1. Open up three terminal windows. In the first, launch roscore:

   $ roscore

2. In the second, we launch ROS’s joystick launcher, replacing jsX with the joystick device you previously identified:

   $ rosparam set joy_node/dev "/dev/input/jsX"
   $ rosrun joy joy_node

3. Now in the third terminal window, we echo the Joy messages, sent out by the joy_node that we just launched:

   $ rostopic echo joy

4. If all is working, you should see a Joy message print on the console whenever the joystick is moved (or a button is pressed).

Identifying Control Buttons & Axes

We now have the task of identifying the various axis and buttons used to fly the drone. All in all, you will need to identify axes to control the AR.Drone’s roll, pitch, yaw-velocity and z-velocity; and buttons to trigger an emergency, command a takeoff, and command a landing.

To identify the emergency button for example, press the button you would like to trigger an emergency, and watch the terminal to see which button is enabled. In my case, I use the joystick’s trigger button for emergency (being able to react quickly to an emergency is very important). With the button pressed, the following is printed in the terminal window:

axes: [-0.0, -0.0, -0.0, -0.0, 0.0, 0.0]
buttons: [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

From this I can identify that: Emergency = Button 0 (note that indexing starts from zero). The process follows similarly for identifying an axis:

1. Press the joystick fully left. The axis that changes  (the first number in the list is axis 0) is the roll axis, the number if changes to is your roll scale (usually + or -1)
2. Press the joystick fully forward. The axis that changes is your pitch axis, and the number it changes to is the pitch scale (usually + or -1)
3. Rotate your joystick fully counter clockwise, the axis that changes is your yaw axis and the number it changes to is the yaw scale (usually + or -1)
4. Push your throttle stick fully forwards. The axis that changes is your z-axis and the number it changes to is the z-scale (usually + or -1).

axes: [1.0, -0.0, -0.0, -0.0, 0.0, 0.0]
buttons: [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]

From this I identify that the Roll Axis = 0 and Roll Scale = 1.

After you have identified all axes and buttons, press Ctrl-C in all terminal windows to stop the processes.

Configuring the Joystick Controller

In ROS, a “Launch File” is used to configure and launch a set of nodes. We’ve already seen this in the first tutorial (and at the beginning of this one), where we typed:

$ roslaunch ardrone_tutorials keyboard_controller.launch

In this command, keyboard_controller.launch configures the ARDrone driver for indoor flight, then launches both the driver and keyboard controller. Similarly in this tutorial, we will be launching the joystick controller (and related nodes) using a launch file.

Go ahead and open a terminal window, then type:

$ roscd ardrone_tutorials
$ geany launch/joystick_controller.launchtemplate

This will open the joystick_controller.launch file in the text editor (non ARDroneUbuntu users can use a text editor of choice). You will need to customize this file for your setup.

• Update line 21 to reflect your joystick device (the /dev/input/jsX that you identified earlier).
• Update lines 27-29 to reflect your button mapping
• Update lines 32-35 to reflect your axis mapping
• Update lines 38-41 to reflect your scales (usually either 1 or -1)
  These scaling factors determine for example the drone roll direction when the joystick is tilted along the roll axis. If your drone does the opposite of what you expect, change the sign on the relevant scaling factor (eg 1 becomes -1)

Feel free to have a look at some of the parameters used on the drone. The drone’s control parameters can potentially be increased to allow for more aggressive flight.

Once you’re happy with the settings, save this file as joystick_controller.launch, then close the geany application.

Finally flying!

So, after that rather long endeavor (which you should only need to do once), lets fly!

Connect to the drone, disconnect other network devices, then open a terminal and type:

$ roslaunch ardrone_tutorials joystick_controller.launch

This will launch the nodes as you configured in launch file. If all goes well, you should now be able to fly the drone using your joystick.

Before commanding the drone to take-off, you should first check that your emergency button works. With the drone still landed, press the emergency button once. All onboard LEDs should turn red. Pushing the button again will reset the drone, and the LEDs will once again turn green. In addition to this, whenever a (mapped) button is pressed, the joystick_controller will print a message to the terminal, such as “Emergency Button Pressed”. This is a further indication that things are working!

You’re now ready to take off. Have fun!

Going Further

The above child’s play? Heres some ideas for exploring the ROS and ARDrone environment:

• While the joystick controller is running, open a new terminal window and type:

  $ rxgraph

  This will display the node communication diagram that we investigated at the start of the post

• Take a look at the data being sent back from the drone:

  $ rostopic echo /ardrone/navdata

• Plot your flight speeds (or any other data of interest) with:

  $ rxplot /ardrone/navdata/vx,/ardrone/navdata/vy

• Adjust drone parameters in the joystick_controller.launch file to fly more aggressively

Troubleshooting

• The joystick doesn’t do anything
  Check that the joystick device you configured in the launch file corresponds to your joystick. To do this, repeat the section “Testing the Joystick”
• My joystick device is correct, but the joystick doesn’t do anything
  Assuming you’ve gone through the “Testing the Joystick” section successfully, launch the joystick controller again and in a separate terminal, type:

$ rostopic echo joy

Assuming everything is configured correctly, this will print joystick messages to the terminal when the joystick is removed.

• If nothing is printed, check to make sure you’ve configured the correct joystick device in the launch file, and that the joystick permissions are correct (see “Testing the Joystick”).
• If something is printed, check to make sure your axis and button mappings in the launch file are correct. If this is the case, leave a comment to this tutorial and I will be happy to help!

• The drone does the opposite of what I expect
  For example flying backwards when the joystick is tilted forwards. To solve this, determine the axes that respond counterintuitively then invert the sign of the respective scaling factors in the launch file. For example, if the pitch axis (forward-backward) is the opposite of what you would expect, the ScalePitch parameter would change from 1 to -1.
• I cannot connect to the drone
  Check that your computer is connected to the drone via wireless, and that all other network devices are disconnected
• The drone will not take off

• Are all the onboard LEDs red? If so, try pressing the emergency button. Do they toggle to green? Perfect – you’re ready to fly!
• If the LEDs are green and the drone still won’t take off, check that your button mapping is correct in the launch file

• Other issues?
  If you have any questions, or comments about what you would like to see in the next tutorials, please send me a message or leave a comment to this post!

You can see all the tutorials in here: Up and flying with the AR.Drone and ROS.

If you liked this article, you may also be interested in:

• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Classroom robotics: Motivating independent learning and discovery

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

In this tutorial (#1) we will:

1. Install ROS, the AR.Drone driver and AR.Drone keyboard controller
2. Fly the AR.Drone using the provided keyboard controller

In tutorials 2-5, we will:

2. Modify the keyboard controller to work with a joystick or control pad, giving much finer control (link)
3. Look more closely at the data sent back from the drone and play with the onboard tag detection (link)
4. Write a controller for the drone to enable tag following
5. Write a controller for the drone which allows us to control drone velocity, rather than body angle

Updates:

Based on feedback I received from a video hangout with Robot Garden (cofounded by fellow Robohub author, Andra Keay), I have made a few updates to the tutorial, summarized below and highlighted throughout the tutorial in red:

1. Fixed permissions problem which caused the keyboard button on the launch bar to do nothing
2. Added keyboard_controller_with_tags.launch file, which enables the drone’s tag detection feature
3. Updated tutorial with command-line equivalent commands, for all graphical actions
4. Included a “Going Further” section at the end of the tutorial

If you are running the ARDroneUbuntu virtual machine, updating the tutorial will download the above updates and fixes.

Command-line equivalent (updating the tutorial):
$ roscd ardrone_tutorials
$ git pull

What is the AR.Drone?

The AR.Drone is a quadrotor that can be flown from any WiFi enabled device with an appropriate application – most commonly an iPhone or Android phone.

Unlike other quadrotors, which can be difficult to fly manually, the AR.Drone is equipped with onboard sensors and controllers that stabilize the vehicle while flying and give it the ability to hover on a spot and await the next command. Because of this onboard stabilization, the AR.Drone is becoming very popular as both a toy (it can be flown by even the most inexperienced pilots), and as a research platform for computer vision and robotic exploration (researchers can focus on their algorithms without worrying that the drone will crash).

And what is ROS?

ROS, the “Robot Operating System”, is a software framework that enables the development of robotic applications – applications that control and interact with robots. A large number of groups using the AR.Drone for research purposes, use ROS to control the drone, which is why we are focusing on it here.

What you’ll need for the tutorial

To complete this tutorial you will need:

1. An AR.Drone!
2. A computer or laptop with WiFi

For future tutorials, you will need:

1. USB Joystick or control pad  (linux compatible)

Getting started

For these tutorials, we will be running the ROS software and AR.Drone controllers using the Linux operating system.

If you don’t have Linux, want to get flying quickly or don’t know what I’m talking about:

To get started, we’ll be downloading an “Xubuntu Virtual Machine”, which has been created especially for these tutorials. This virtual machine package contains everything that you need to get up and flying – fast!

To run the virtual machine package, you will first need to download and install the free Virtual Box software. You will also want to download the ARDroneUbuntu virtual machine package.

Once you have Virtual Box installed, double click the ARDroneUbuntu.ova file that you downloaded. This will open it in the Virtual Box program.

A settings/import dialog should open. You should leave these settings as they are, unless you know what you’re doing.

After completing the import process, click start in the Virtual Box program to begin running ARDroneUbuntu.

Congratulations, you’re now running Xubuntu 12.04 (username: ardrone, password: ardrone)!  This has been preconfigured with all the software required for this series of tutorials, so you’re now good to go!

Once the virtual machine is running, you will be brought to the desktop. At the bottom of the screen you will see the launch bar:

The launch bar contains a number of icons, which I will be referring to throughout the tutorial:

1. Terminal – in this program, you run text-based programs and commands
2. Text Editor – we will be using this in future tutorials to write some basic programs
3. Keyboard Controller – this launches the keyboard controller
4. Update Tutorials – this will update the AR.Drone driver and tutorial source code, it will also install new tutorials once they have been released.
5. Quick Command Box – this is like the terminal and can be used for sending a quick command to the operating system

Great, now that you’re up and running, lets update the tutorial files. Click the Update Tutorials (#4) button in the launch bar. This will launch a terminal window in which it will download and install any required software updates. You should be connected to the internet for this step.

Command-line equivalent (updating the driver):
$ roscd ardrone_autonomy
$ git pull
$ rosmake ardrone_autonomy

Command-line equivalent (updating the tutorial):
$ roscd ardrone_tutorials
$ git pull
$ rosmake ardrone_tutorials

Now you’re ready to fly and can skip the next section of the tutorial.

If you have Linux (skip this section if you just downloaded the virtual machine):

I’ll assume you understand enough about linux to follow the ROS installation instructions (summarized here). I also assume you are running Ubuntu (or a deviant). If not, you should modify the apt-get commands to suit your system.

In addition to the summarized ROS installation instructions above, you will need to run:

$ sudo rosdep init
$ rosdep update
$ sudo apt-get install ros-fuerte-joystick-drivers
$ rosdep install joy

This will install the joystick drivers which are used in a future tutorial.

Now to install the AR.Drone driver’s prerequisites:

$ sudo apt-get install daemontools libudev-dev libiw-dev

Next, we will download the AR.Drone driver (full disclosure: I am a regular contributor to the driver) and the keyboard controller used for this tutorial:

$ roscd
$ pwd
~/ros_workspace
$ git clone https://github.com/AutonomyLab/ardrone_autonomy.git
$ git clone https://github.com/mikehamer/ardrone_tutorials.git
$ ls
...  ardrone_autonomy  ardrone_tutorials  ...
$ rospack profile

And now to compile the packages:

$ roscd ardrone_auto<TAB should autocomplete>
$ ./build_sdk.sh
$ rosmake -a

If TAB does not autocomplete the above-indicated command, or if rosmake reports any failures, you may have a problem with your installation. If this is the case, you should check the installation guide linked above.

If you’ve completed all of the above successfully, you’re ready to move on!

Warning!

We are now going to fly the drone. Flying inside can be dangerous. When flying inside, you should always use the propellor-guard and fly in the largest possible area, making sure to stay away from walls and delicate objects. Even when hovering, the drone tends to drift, so always land the drone if not being flown.

I accept no responsibility for damages or injuries that result from flying the drone.

If you have a laptop and are thus able to fly outside, check the last section of this tutorial for instructions on reconfiguring the drone.

First connection

Great, we’ve got the software, now lets have some fun!

Turn on the drone and wait until the propellors vibrate, indicating that the drone is ready.

Now connect your computer’s wireless to the AR.Drone’s network, just like you would connect to a normal wireless network. The network will be called ardrone<something>.

If your computer is also connected to a network via ethernet (or any other network adaptor), disconnect it. We only want the wireless network to be running*

* Advanced users: The drone acts as a wireless server and assigns itself (via its DHCP server) a fixed IP address of 192.168.1.1 and your wireless card an address from 192.168.1.2-255. For this reason, there can be routing issues if your ethernet card is also in the 192.168.1.0 subnet.

Running the keyboard controller

If you’re using the supplied virtual machine:

Click on the keyboard icon (#3) in the launch bar to run the keyboard controller program:

Note that a connection can take up to 10 seconds. If after 10 seconds the window still says “Disconnected”, check the wireless connection to the drone, and check that any network cables have been unplugged from the computer.

If you’re using a custom Linux installation or want to use the command line:

Open a terminal window and type the command:

roslaunch ardrone_tutorials keyboard_controller.launch

Pressing TAB at various stages throughout the command should autocomplete. If this is not the case, or if this command causes errors, check that you have setup your .bashrc file as per the ROS installation guide, and that you ran rospack profile after cloning the two git repositories.

If successful, this command will print out many startup messages, including:

Exception AttributeError: AttributeError("'_DummyThread' object has no attribute '_Thread__block'",)...

This exception is a Python bug and won’t affect the program.

You will also see a printout:

process[ardrone_driver-2]: started with pid [6178]
===================+> 192.168.1.1

Within a few seconds, the output should progress well past this stage and begin printing configuration messages as the initialization of the AR.Drone progresses. If this is not the case, check the wireless connection to the drone, and check that any network cables have been unplugged from the computer.

Up, up and away!

If the keyboard controller launched successfully, you will now be seeing a live video stream from the drone. If this is the case, we’re ready to fly!

The controls (defined in ardrone_tutorials/src/keyboard_controller.py) are:

• E – Pitch Forward
• D – Pitch Backward
• S – Roll Left
• F – Roll Right
• W – Yaw Left
• R – Yaw Right
• Q – Increase Altitude
• A – Decrease Altitude
• Y – Takeoff
• H – Land
• SPACEBAR – EMERGENCY STOP

Update 17/12:
Note: Pressing the Spacebar will toggle the drone between an emergency state (all onboard LEDs are red), and the ready state (all onboard LEDs are green). If your drone isn’t responding to commands, make sure it is in the ready state!

With that said, I now hand the controls to you. Remember to keep a thumb on the Spacebar at all times, just in case something goes wrong.

Have fun!

Going Further (Added 17/12/2012)

This tutorial was fairly straight forward, the purpose being simply to get you up and running with the right software. Future tutorials will be more technical, however if you’re wanting to scratch the itch, try running some of the following commands at the terminal:

With a controller running:

$ rostopic echo /ardrone/navdata

Will print the data packets that are being sent back from the drone at 50Hz.

$ rosservice call /ardrone/togglecam

Will toggle the drone’s video stream between front and bottom cameras.

$ rxplot /ardrone/navdata/ax:ay /ardrone/navdata/vx:vy

Will plot the x and y acceleration on one plot, and the estimated x and y velocity on a second plot.

Now without a controller running (ie close the keyboard_controller window)

$ roslaunch ardrone_tutorials keyboard_controller_with_tags.launch

This runs a new launch file that enables tag detection. Now run:

$ rostopic echo /ardrone/navdata

And have a look at the various tag detection data that is returned (you have to be holding tags in front of or below the drone for this to work).

By default, this launch file configures the drone to detect Orange-Blue-Orange stickers on the front facing camera, and the A4 “Oriented Roundel” that comes with the AR.Drone 2.0 on the downward facing camera.

If your stickers are a different color, change the launch-file parameter enemy_colors to 1 for Orange-Green-Orange, 2 for Orange-Yellow-Orange and 3 for Orange-Blue-Orange.

Coming up in tutorial #2 – Joystick control

By now you may have noticed that flying with the keyboard isn’t so easy. This is largely due to the binary nature of a keypress (not pressed/pressed), which translates into 0%/100% control. Although we could implement software remedies such as input windup based on how long the key has been held, this is for the most part only a band-aid solution; what we really need is an analogue input device. Enter the Joystick (or control pad).

In the second tutorial of this series, we will be covering a little ROS theory (topics, messages and launch files), before installing a joystick, linking it up to the ROS system and then using it to fly the drone.

Source code and configuration

For those of you interested in the source code or configuration, I will briefly discuss the various options below. We will be going into much more detail in the next tutorial.

Full source code for this tutorial is available from the GitHub repository.

src/drone_controller.py

This file provides an abstracted interface to the AR.Drone driver and shouldn’t need to be modified

src/drone_video_display.py

This file implements the video display window and shouldn’t need to be modified

src/keyboard_controller.py

This file brings everything together. The KeyboardController class implemented in this file extends the DroneVideoDisplay class, adding a keyboard handler for control. Key mappings are defined in this file and can be changed to suit your preferences.

launch/keyboard_controller.launch

This is the default configuration file, which sets the drone’s configuration and launches all required processes

launch/keyboard_controller_outdoor.launch

This configuration file implements a more aggressive set of flight parameters for outdoor flight. If you are wanting to fly outdoors, the keyboard controller can be launched with the command:

roslaunch ardrone_tutorials keyboard_controller_outdoor.launch

For more information about configuring the AR.Drone, have a look at the AR.Drone driver’s readme.

That’s all folks

I hope you’ve had fun with this tutorial and I look forward to bringing you the next one in two weeks time!

If you have any questions, or comments about what you would like to see in the next tutorials, please send me a message or leave a comment to this post! You can see all the tutorials in here: Up and flying with the AR.Drone and ROS.

If you liked this article, you may also be interested in:

• Parrot AR.Drone app harnesses crowd power to fast-track vision learning in robotic spacecraft
• Taking your first steps in robotics with the Thymio II
• Discovering social robotics with the Aisoy1
• Classroom robotics: Motivating independent learning and discovery

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.