Occupational Therapist Rosalie Wang investigates how intelligent haptic robotic systems can benefit people recovering from upper limb disability due to stroke. Photo courtesy IATSL (Intelligent Assistive Technology and Systems Lab) at Toronto Rehab, University of Toronto.

Robotics has always been an interdisciplinary field – one that integrates knowledge from computer science, mechanical, electrical, controls, and other areas of engineering. But as robots move out of factories and research labs, and into our homes and workplaces, another breed of robotics expert is emerging – and an engineering or computer science degree is not necessarily part of their resume.

From occupational therapy to rehab robotics

Rosalie Wang is Assistant Professor at the Intelligent Assistive Technology and Systems Lab (IATSL), a multi-disciplinary research group at the University of Toronto made up of engineers, computer scientists, rehabilitation and medical researchers who leverage artificial intelligence and robotics to develop intelligent assistive and therapeutic devices. IATSL research runs the gamut, including obstacle avoidance and guidance systems for powered wheelchairs, fall detection solutions, intelligent haptic systems for stroke rehabilitation, personal robots to assist aging-in-place, assistive robots for people with dementia, and much more.

But Wang started her career working as an Occupational Therapist, supporting clients in long-term care. Her passion was helping seniors, and it was while working at a nursing home that she found herself tinkering with her clients’ wheelchairs. “I would come home with grease in my fingernails, having taken apart and put back together a wheelchair so that it would better fit my client’s needs.” One thing led to another, and soon she found herself doing a PhD on collision-avoidance technology for powered wheelchairs.

Translating tech into practice

Finding technical solutions to ‘people problems’ is a special skill that requires deep knowledge of both the end-user and their context, and an ability to work closely with engineers and computer scientists to understand the possibilities and limitations of the technology.

A common problem with powered wheelchairs, says Wang, is that they can be hard for someone who has physical and/or cognitive impairments to control; and because they are powered, they can also be more dangerous than manual wheelchairs, especially to someone who is already in a fragile state of health – one reason why some nursing homes have banned the use of powered wheelchairs altogether. And then there are the ethical aspects: “It’s hard to take a powered wheelchair away from someone once you’ve given them one. We want to enable mobility, and we want to keep people safe, and we want to do it in a way that respects the client,” says Wang.

Wang also points out that simply having technology that works under certain conditions doesn’t mean it will be taken up in practice; many contextual factors that lie outside the technology itself can get in the way. For example, most fall detection monitoring services tend to use a subscription-based business model, similar to home monitoring. But Wang has found that many clients prefer to have their personal care monitoring service trigger warning calls to a family member rather than a call center – even though this solution is less likely to be taken up by industry partners. “As you’re designing the system, you have to look at the whole picture. Clients feel there is a stigma attached to disrupting a call centre.”

Developing technology into systems and products that can actually be deployed into real world scenarios is a challenge, and it’s important to understand clients’ personal preferences and their physical and cognitive abilities, as well as contextual factors, such as nursing home architecture, routines and regulations, family and caregiver needs, ethics and more. This is why a critical part of the research at IATSL is also to develop frameworks, guidelines and methodologies for applying intelligent assistive technologies in nursing homes and at home.

Rosalie Wang speaks about her career path from Occupational Therapist to rehab robotics researcher at the Women in Robotics lecture series, hosted by Get Your Bot On and Women Engineers TO in Toronto, Canada. Photo via Nicole Proulx.

For Wang, the benefits of being an Occupational Therapist in the field of robotics and AI cut both ways: she can bring a user-centered perspective to designing the technology, but “Robotics and AI can also help to augment the work of the Occupational Therapist and advance the field as a whole,” she says. Wang’s research also includes clinically evaluating robots for upper limb stroke rehabilitation and assistive robots to help older adults with dementia. Her latest paper, just published last month, examines how older adults with Alzheimer’s and their caregivers feel about using robots to help them complete tasks at home, and is one of the few studies to give an in-depth perspective on the issue.

From an Occupational Therapist’s perspective, one of the key benefits of employing robotics and AI is that it can yield data for evidence-based therapy. “It used to be that we could only measure therapy results with very coarse snapshots … a client either could complete a task or they could not complete it. This kind of feedback is potentially very demoralizing to the client, and makes it harder for the therapist to perceive and record small changes in progress… Now we can collect continuous data, which lets us design more refined therapies, and gives clients a better sense of progress. A therapy robot can give you continuous clinical data, even when you can’t perceive it.”

When asked if she ever imagined she would be working in robotics one day, Wang gave a resounding answer: “No.” And she is reluctant to describe herself as a roboticist. Still, given the wide range of projects being undertaken at the Intelligent Assistive Technology and Systems Lab, it’s clear that it takes an interdisciplinary team with all kinds of expertise to translate robotics and AI into practical healthcare solutions.

Team members at IATSL have backgrounds that include occupational therapy, neuroscience, industrial design, biology, clinical psychology, experimental psychology, rehabilitation science, and nursing.

If you drive a car, it makes little difference what brand it is: all cars are driven in essentially the same way. The same applies to computers. If you have a Windows PC, the user interfaces won’t be affected by your computer hardware. But this is definitely not the case for industrial robots.

The way you program an industrial robot strongly depends on the robot brand. You might think that a robot program looks like C++ or Visual Basic. The truth is that some robots are still programmed in assembler-like languages.

Because of the complexity of using an industrial robot, robot integrators and most manufacturers try to limit themselves to a single robot brand. Unfortunately, industrial robot manufacturers don’t offer completely equivalent product ranges, and their offers evolve constantly. Stäubli, for example, is one of the few manufacturers that offers robots suitable for humid environments, while FANUC is the only one to offer a medium-payload collaborative robot. Furthermore, a number of businesses work with calls for tenders. As a result, sometimes manufacturers end up with different robot brands.

The reason why robot-programming languages evolve slowly is probably due to the fact that industrial robots are extremely reliable and durable. Thus, robot manufacturers have to offer backward compatibility.

Further, selling spare parts and additional options is probably as lucrative as selling new robots. For this reason, robot manufacturers probably won’t change the fundamentals of their programming languages. For example, ABB is stuck using quaternions to represent orientation, even though a quaternion is as intuitive as using binary code instead of ASCII. Rather than changing quaternions with Euler angles (like every other robot manufacturer), ABB simply added functions that convert from one convention to the other. Robot programming languages probably looked fine a decade or two ago, but nowadays, engineers are used to modern programming languages like Java, C#, and Python.

FANUC, one of the leading four robot manufacturers, provides not one but two different programming languages: Teach Pendant (or TP) and KAREL. TP programs are binary files that can be edited through the robot’s teach pendant buttons (or touch screen for newer robots). TP files can also be compiled/decompiled from an LS file (human-readable ASCII file) TP programs offer a limited assembler-like functionality. Alternatively, with FANUC, you can program your own algorithms using a PC and KAREL (programming language based on Pascal), but KAREL does not allow you to do robot movements nor edit the program from the controller’s teach pendant.

You can purchase ROBOGUIDE off-line programing and simulation software for about US$2,500/year to off-line program FANUC robots. However, if you don’t have ROBOGUIDE’s WinOLP, you won’t be able to use LS programs unless you also have the option “ASCII Upload” in your FANUC robot controller, a paid software upgrade required for each robot. You also need ROBOGUIDE to edit and create KAREL programs.

Motoman robots are programmed using the INFORM programming language (JBI files). INFORM looks similar to Fanuc’s LS programs. The program variables are registers shared among all programs. However, Motoman also offers MotoCom for all their robots: a well-documented library that allows you to program the robot using C++, C# or Visual Basic (on Windows).

Programming ABB robots is simpler, and based on the RAPID programming language, which looks like Visual Basic. Furthermore, ABB provides RobotStudio, offering on-line programming and an excellent RAPID program editor for free as well as an easy file transfer through a network connection between a PC and the robot. Optionally, you can pay about US$1,500 US/year to have the simulator and offline programming tools.

KUKA robots require SRC programs written in their KRL programming language, usually edited using the free OrangeEdit editor unless you have the KUKA simulator Sim Pro (priced at about US$2,000). It is interesting to note that Kuka’s KRC4 controller is a Windows based computer and the teach pendant is a remote desktop of that computer. It is unpractical to modify programs from the teach pendant as the touch screen feels slow. One notable exception, however, is KUKA’s collaborative robot, LBR iiwa, which uses KUKA’s Sunrise controller and is programmed in Java.

We’ll stop here, because there are dozens of other robot manufacturers from all over the world. Some brands do not have offline programming software (such as Universal Robots); the user is then forced to program robots through the robot controller’s touchscreen (which is highly intuitive, but also very limited) or using an ASCII editor and their UR Script language (based on Python). Other robot manufacturers focus their efforts on having plugins for CAD software, like Mitsubishi who offers MelfaWorks for SolidWorks.

The fact that nobody has taken a step towards unifying the way robots are programmed is very disappointing. G-code was introduced in the 50s, shortly after Numerical Control, and most CNC controllers currently use it. No such common programming language exists for industrial robots.

A few companies offer robot off-line robot programming tools that support multiple robot brands, such as Octopuz, Delmia, Siemens NX, and RobotMaster. However, the price of these software tools ranges between US$15,000 and US$100,000 per license. Clearly, these solutions are too expensive for most small and medium sized companies. Furthermore, some of these software tools target very specific applications (e.g., machining only). The market of used industrial robots is growing steadily and it is now easier to find robots at affordable prices. Some robots still work perfectly even 25 years after they were manufactured. You can currently find used six-axis robots for only US$10,000, but you can’t buy second-hand off-line robot programming tools.

This is why we created RoboDK: a simulator for industrial robots that provides a new way to program industrial robots, more intuitive off-line programming and more affordable. The simulator supports a wide variety of applications, such as pick and place, painting or robot milling (optimization tools are provided to automatically convert CAM programs to robot programs). RoboDK is a spin-off from the CoRo laboratory, one of the largest robotics labs in Canada, where it was tested on all major industrial robot brands.

RoboDK’s library has more than 200 robots from 15 different robot manufacturers and we look forward to continue adding support for new robot brands. Any robot manufacturer is welcome to include their robots in RoboDK’s library, providing an open interface to be able to do off-line programming and online programming as well. Off-line programming is achieved by defining the robot’s brand post processor. The online programming is achieved by developing a robot driver that works as a bridge between the simulator and the robot. Then, it is possible to easily run robot programs step by step, retrieve the robot position or move to a specific position with a robot. This video shows an example with an UR10:

What makes RoboDK unique is that you can program any robot using Python and the RoboDK API. Python offers an immense library of tools for programming (for example, to convert SVG images to XY coordinates that can be followed by robots). You can then use object oriented programming with Python and RoboDK’s API and have code snippets like:

# robot is an object
# target is a pose (4x4 matrix), defining position and orientation of the TCP
robot.MoveJ(target*transl(0,0,-100))    # Move (joint) to the approach position
robot.MoveL(target)                     # Move (linear) to the target
CloseGripper()                          # Custom-defined action, such as grabbing a part
robot.MoveL(target*transl(0,0,-100))    # Move back to the approach position

… which will move the robot to a specific target through an approach position. Many examples are available in our library (with and without Python).

There is no doubt that one day a robot programmer will have the choice to program any robot brand using a universal programming language such as Python. Robot programmers as well as companies prefer this option as they do not depend on brand-specific tools. Kudos to Mecademic, another spin-off of CoRo, who offers the first industrial robot arm that can be controlled with any modern programming language.

RoboDK has been downloaded more than 2,000 times since we launched it in January 2015. RoboDK has also attracted universities as a tool for learning industrial robotics as well as new robot manufacturers all over the world. We also welcome integrators to develop their technology using RoboDK’s API. The advantage is that they can develop their technology with one robot brand and deploy it with any other robot brand.

RoboDK can be tried for free, it will remain free for educational purposes and is the most affordable offline programming solution available.

Via Wikimedia Commons

In episode twenty one  we talk with Quaid Morris of the University of Toronto, who is using machine learning to find a better way to treat cancers. Ryan introduces us to expectation maximization and we take a listener question about how to master machine learning.

Launched in 2009 by a group of Waterloo engineering students, Clearpath‘s unmistakeable bright yellow and black robots have become synonymous with unmanned vehicle research in university research labs around the world. Now, as the field of robotics matures, Clearpath is forging into industrial applications, too. We caught up with CTO Ryan Gariepy at the 2015 Field and Service Robotics conference (FSR), to talk about their roots in research, the role of ROS and open source in their business model, and the challenges and opportunities of launching a robotics startup in Canada.

[HS] You got your start in the business of robotics while you were still a student at Waterloo. What made you decide to launch a robotics company at that time? Did you have a specific project or technology you were developing that made you think, “Hey, this could actually be useful out there!” …? Or was it that you saw the potential in the field more generally?

[RG] We started Clearpath right out of university in 2009 as we finished our undergraduate degrees – I did my Masters at the same time as we were getting off the ground, which is generally regarded to be a bad idea in terms of stress levels. It’s hard to believe that we just celebrated our sixth anniversary this year.

Starting the company was less about the technology and more about the potential … we just looked at everything around us and said, “Why do we have to shovel snow? This can be automated. Why do we have to do any work that does not require human creativity?” There are a lot of dangerous jobs out there that most people don’t realize are performed on a daily basis by tens of thousands of people in order to keep society going, like mining, farming, construction, stocking warehouses. And yet we need fuel and food and buildings, and we want our same-day deliveries from the Internet. There is a demand for this kind of work to take place.

Around the time we were starting out, Google Maps was getting off the ground. The concept of having information available at your fingertips was also becoming compelling and valuable to people, and the best way to get information about the world on a constant basis is to involve robotics or automation in some form. At the same time, sensors and the cost of sensing were improving due to the rapid uptake in smart phones; we were seeing advances in laptops, smartphones, and electric and hybrid vehicles driving down costs and improving battery density.

People now accept that robotics, automation, and intelligent systems can actually solve some of their problems.

All of this was coming together and we felt that autonomous robotics was possible at a relatively high reliability and low cost, especially when compared with what had been done before.

What would you say have been the biggest changes that you’ve seen in the field of robotics since then?

I think one of the biggest changes is simply that people now accept that robotics, automation, and intelligent systems can actually solve some of their problems.

This recognition means that there are a lot of opportunities out there, both in terms of hardware and software. In some industries – ones that have been around for a long time – they already know how to build solid hardware that could be made robotic, but they need the right parts and the right software to automate it. Then there are other industries that need full solutions. There are a lot of pieces starting to come together now.

Your early customer base was made up of mainly universities and research labs. Do you find yourself getting pulled into a different direction now that you’re expanding more into industry?

The research market is definitely maturing and our product development cycle within that side of the business is very well understood. We’ve released two completely different platforms in the last eight months alone.

Now we are in a position where we can leverage the experience we gained from the R&D market to address challenges in industry. We are actively pushing the development of full robotic solutions (including software), and will soon be releasing a new platform along these lines.

How do you manage the problem of building in more than one direction at the same time?

One of our biggest challenges is that there is no one single clear challenge, especially at the scale we’re at right now. At this stage there are many things that need to be focused on: sales, operations, quality assurance, new features, understanding where the market’s going, product management, marketing … so the challenge is more about managing the priorities of any given department.

How do I provide guidance to the team so that they can spend their time on the right tasks? How do we remain dynamic as a startup? How do we remain flexible enough to change direction quickly if we need to, but still be stable enough to stick to our core values? These are the kinds of questions we are dealing with.

Speaking of core values, Clearpath took a chance from the beginning by focusing on open source and ROS. Does open source make for a good business model?

Why open source? No matter how large a company we become, there will always be more smart people who can build and program robots outside our company than within it.

One of the reasons we looked at open source was the philosophy that no matter how large a company we become, there will always be more smart people who can build and program robots outside our company than within it. So it’s only logical that we work with whatever the community ends up creating around open source robotics, both to reinforce the community, and to hire from it.

That said, from a business model perspective it is more complicated. There are many companies that succeed with open source, but there is still no clear-cut model for building an open source company – and that tends to make people nervous. And yet by now we’ve seen several companies build on open source, including Google and Facebook, and it’s the same thing as building a startup on a LAMP stack: yes, you use open source and you may push back some code to open source, but you still have value you’re bringing on your own.

Earlier on we sold only hardware, which is easier to protect. Now that we’re getting into software sales, open source creates additional considerations for us, but it still hasn’t been an issue. Building on open source makes us more effective, and it certainly helps us engage much more with the robotics community.

Now that you’ve done all the legwork of developing the product and proving the market, do you have any concerns that another company with more resources could just come along and say, “We know who their buyers are, and we know what their markets are, and we can imitate their products …”?

Clearpath is sponsoring the 2015 Robot Launch “Open Source Robotics” Award, offering 10 hours of ROS consulting services to the winner.

ROS is open source and we contribute a lot back to it. We test a lot, we support people, we contribute drivers, and we contribute utilities back. However, the work that we do on autonomy and in controls –planning, mapping localization, interfacing, vehicle management – is not open source. We may open source parts of it in the future as we advance our state of the art, but the ‘secret sauce’ is still closed, even though it’s built on ROS middleware.

Could it make life easier for our competitors in some cases? Yes, but by the same token nobody makes money on having a better robot-logging framework, for example, and everybody saves money by having a good free robot-logging framework. If a better logging framework is your only competitive advantage, it’s not really an advantage, is it? Personally, I’d rather not spend money on a logging framework, and spend more to compete on the actual value that we bring to our clients.

You mentioned snow shoveling as one of your inspirations, which is a stereotypically Canadian problem. Did the Canadian climate and landscape figure into your business plan from the beginning? Is that why you wound up in field and service robotics, as opposed to some other application area?

Of course our robots can survive the winter – we’re Canadian!

One of our competitive advantages is the terrible weather we have in Canada. We have harsh winters, we have a lot of forestry, a lot of primary resources to mine and harvest, and we have a lot of space that is sparsely populated, which makes it easier to do testing. We also happen to have a very well developed technology ecosystem here. When you put those things together, it’s natural to lean towards field robotics.

When we started there were very few platforms out there that could actually do field robotics, and so we set that as our first niche.

How is the Canadian business landscape for robotics?

Many of our first customers were Canadian universities, some of which are leading the way in robotics integration and research. Meanwhile, Industry Canada is looking at how robotics technology can be adapted to existing manufacturing clusters in Southwestern Ontario, like the auto sector, which took a hit when the North American auto industry had a downturn. Canada also has strengths such as communications and optics, which naturally extend into robotics. It also helps when we can do business with Canadian suppliers; we get our GPS’ from NovAtel, which is based in Calgary, for example.

That said, as a Canadian company you’re forced to go multinational almost instantly because there isn’t enough of a robotics market in Canada to stay Canadian for long. We recently announced that we’ll be expanding office space into Silicon Valley and a primary reason for that is because of the opportunity that is available in the US market. Regardless, the Waterloo Region will continue to be Clearpath’s ‘home base’ for the foreseeable future.

People now refer to Waterloo as ‘Silicon Valley North’. What’s changed in the incubation space where Clearpath grew up since you launched?

It used to be that investors would demand that you move to the area where their office was, which was typically Boston or New York City or Silicon Valley, but that’s not always the case anymore.

It used to be that investors and venture capitalists would demand that you move to the area where their office was, which was typically Boston or New York City or Silicon Valley, but that’s not always the case anymore. Investors are starting to recognize that there are also benefits to being in other locations, especially for startups. Waterloo has a very dynamic tech community. It’s also a quarter of the price to buy a house here than in some of those larger US cities, and the time it takes me to get to work is 10 minutes, which makes it pretty livable and affordable compared to Silicon Valley. We found that VCs are now willing to accept that good ideas and businesses can grow and thrive in other areas as long as these areas have the right attitude – and Waterloo certainly has that.

Take the displacement of BlackBerry employees for instance – quite a few very smart people who were working at Blackberry have moved on to bring their skills and experience to other industries. We’ve hired a number of them, and a number of them have gone off to launch their own startups.

On top that is the continual increase in the brand of the Waterloo Region, and specifically the University of Waterloo (UW), as the place to go for getting a great education and practical experience that can lead to entrepreneurship. YCombinator’s Paul Graham has noted that UW has a fantastic record for turning out very strong companies, specifically hardware companies.

The other change worth mentioning is that Clearpath isn’t alone. The first group of startups that came out of UW includes Pebble, Vidyard, and more recently Thalmic Labs. These are all people who have graduated from UW’s engineering program within a few years of each other.

Does that help, too? Having a cluster that’s grown up together?

It certainly does.

Take recruiting, for example. When you’re hiring someone it helps prospective employees to know that there are other options. Clearpath has hired quite a few people from outside the region and if you’re taking a risk and moving a few thousand kilometers to join us, it’s good to know that there are other companies around, whether it’s for future growth opportunities, shared interests, or participating in knowledge transfer. For prospective employees, knowing that you can get back on your feet if you need to, or that your partner can find a job in the region, is very useful.

There is also something to be said for being part of a startup community. We all know each other, and we try to mentor each other. Even if you’re not working in exactly the same market, you can look at other startups and see them succeeding – and this is evidence that you can do it too.

Tethered Robotic Explorer (TReX)

Autonomous Space Robotics Lab (Tim Barfoot), University of Toronto

June 26, 2015 – Patrick McGarey unveiling the new Tethered Robotic Explorer (TReX) robot to a group of attendees at the Field and Service Robotics conference (FSR) demo day.  Photo credit: François Pomerleau – University of Toronto.

TReX can climb various kinds of steep terrain with the help of its onboard tether management system. Here it climbs a 60 degree metallic roof of the UTIAS Mars Dome. Photo credit: Hallie Siegel.

June 26, 2015 – TReX climbs a small flight of stairs before entering the Mars Dome at UTIAS. Photo credit: François Pomerleau – University of Toronto.

TReX weighs roughly 90 kg and was built on a Husky from Clearpath Robotics. Photo credit: François Pomerleau – University of Toronto.

TReX build

Grizzly RUV

Autonomous Space Robotics Lab (Tim Barfoot), University of Toronto

June 26, 2015 – Grizzly RUV negotiating a sharp turn on a bumpy terrain composed of sand and gravel. The Grizzly RUV is built by Clearpath Robotics and weighs around 1000 kg. Autonomous driving capabilities were demonstrated over a long distance using only one stereo camera in a GPS denied environment. Photo credit: François Pomerleau – University of Toronto.

Grizzly RUV crossing dense vegetation and reaching autonomously one of its intermediate goals under a blue gazebo. The robot can drive through different types of environments — indoors, outdoors, on roads, and through tall grasses — using the same algorithms. Photo credit: François Pomerleau – University of Toronto.

Robust control for quadcopters

Dynamic Systems Lab (Angela Schoellig), University of Toronto

June 26, 2015 – Robust control under large perturbations was demonstrated using a quadcopter tracked by a Vicon system. Photo credit: François Pomerleau – University of Toronto.

June 26, 2105 – Karime Pereida Pérez launching a flight sequence for an Unmanned Aerial Vehicle (UAV). In the background, Rikky Duivenvoorden awaiting to apply interference forces on the UAV from inside the flying arena. Photo credit: François Pomerleau – University of Toronto.

Hexrotor drone

Waterloo Autonomous Vehicles Laboratory (Steven Waslander), University of Waterloo

June 26, 2015 – The Waterloo Autonomous Vehicles Laboratory hexrotor drone, affectionately named Olga, sports three high-speed wide field-of-view cameras, which it uses to determine its own motion in unknown environments. Olga recently flew in Kelowna, BC at the NCFRN Field Trials, where it used the multi-camera cluster to construct maps of cliff faces and hiking trails, similar to those needed in mining and agriculture inspection applications. Photo credit: François Pomerleau – University of Toronto.

SL Commander

MDA Robotics and Automation

June 26, 2015 – The SL Commander with its sensor suite (laser, stereo camera, mono cameras) on a football field. The SL Commander was built by Bombardier Recreational Products (BRP) under the supervision of MDA. The rover is fully electric and weighs around 1200 kg. The maximal speed (40 km/h) and controllability of the platform were demonstrated by remotely operating the vehicle without a driver. Photo credit: François Pomerleau – University of Toronto.

About FSR

The bi-annual Field and Service Robotics conference is focused on applying robots to real-world problems such as monitoring lakes and oceans, autonomous wheelchairs, automated mining, space exploration, drones, and self-driving cars. You can learn more about FSR and the papers presented at this year’s conference on the FSR Toronto website, or check out the FSR Toronto coverage on Robohub. The next edition of FSR is schedule to take place in Zurich, Switzerland in 2017.

June 26, 2015 – Participants of the 10th Conference on Field and Service Robotics (FSR). The conference hosted roughly 100 participants and 42 original scientific papers were presented. Photo credit: François Pomerleau – University of Toronto.

• Field and service robotics: Live tweets and pics from #FSRToronto
•  #ICRA15 photo essay
• Fukushima photo essay: A drone’s eye view
• Robots Podcast: Field robotics

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Created by Mimetics, the Jade Robot is an innovative, fully assembled robot that is manufactured in Canada. The large OLED display provides an introduction to the robot and configuration menus as well as the ability to create surprisingly sophisticated applications which respond to sensor inputs. The object and light sensors are all-aspect, providing the ability to sample the environment around the robot, not just in front of it. The built in Reflectance Spectrometer provides students with a working instrument to help them understand the challenges faced by NASA with the Curiosity rover on Mars. Programmed through Bluetooth, users can create Scratch and C applications for the Jade Robot.

Despite being fully assembled, the Jade Robot is designed for expansion. For example, it’s got simple add-ons like this pen holder …

… this camera/gripper combination, which uses the Bluetooth link for bi-directional transmission of movement commands, data and video:

… as well as user-designed I²C Expansion adapters, which provide functions like digital compasses, accelerometers and remote weather sensors.

The Jade Robot wasn’t created in a day. As can be seen in the above animation, it went through many iterations to get from miscellaneous PCBs on a sheet of Plexiglas to the robust and rugged robot that has been enjoyed (and abused) by thousands of students.

Along the way, we listened to the input of many, many users. One of the most surprising requests from users was to get rid of the plastic cases that most educational robots have, and provide students with bare PCBs with labeled parts so that the technology is not hidden from them but is something that they can touch and experience.

Please help us crowdfund for the Jade Robot so that we can engage and excite the next generation in STEM!

While there has been a lot of recent attention in the media surrounding self-driving cars and drones, the field and service robotics community has been quietly toiling away for much longer. Later this month, the world’s top experts on field and service robotics will meet to discuss their latest accomplishments at the bi-annual Field and Service Robotics (FSR) conference. These researchers are applying robots to real-world problems such as monitoring of lakes and oceans, autonomous wheelchairs, automated mining, space exploration, and yes, self-driving cars and drones, too. The first FSR was held in Canberra, Australia, in 1997, and it remains the only focussed meeting (single-track) on field and service applications of robots.

This year’s FSR (the 10th edition) will be held in Toronto, Canada, from June 23rd to 26th and the lineup could not be more exciting. There will be 42 papers presented on a wide variety of applications and the keynote speakers are second-to-none: Dr. Chris Urmson (Google[x]), Prof. Paul Newman (Oxford), Prof. Sanjiv Singh (Carnegie Mellon), and Ryan Gariepy (Clearpath Robotics). The full list of talks and abstracts can be found on the FSR webpage.

Science fiction often asks us to imagine what a world full of robots might look like. Well, look no further. The field and service robotics community is approaching this vision at breakneck speed. If you’re interested in previewing your future with robots, register and join us in Toronto for FSR 2015. The robots are coming …

image: Matt Chamberlain.

On May 28^th, Transport Canada released a Notice of Proposed Amendment (NPA) outlining proposed changes to the regulatory framework governing small UAVs. If adopted, the changes would take effect in 2016. The deadline to submit comments is August 28^th.

Minimizing the Application of SFOCs

Transport Canada proposes to establish regulatory requirements for UAVs weighing 25kgs or less (sUAVs) that are operated within visual line-of-sight (VLOS). Operators using sUAVS within VLOS that would be able to meet the regulatory requirements would be permitted to operate without having to obtain Special Flight Operation Certificates (SFOCs). Operators using UAVs weighing more than 25kgs and/or those seeking to operate beyond VLOS would continue to use the SFOC process.

Establishing Risk-Based Categories of Operations

Transport Canada proposes three categories of operations involving sUAVs that are flown within VLOS: Complex Operations, Simple Operations and Operations Involving Very Small UAVs.

Under each of the three categories, operators that meet certain criteria would need to register with Transport Canada and obtain an Operator Certificate by demonstrating that they have an “adequate management structure and can conduct a safe operation”. The criteria being considered includes:

• The number of employees;
• Companies that retain persons in commercial UAV enterprises; and/or
• Companies seeking to operate across multiple regions and/or using a large number or multiple types of UAVs

Such operators would have to meet certain requirements for flight operations, documentation, flight time and duty time limitations, emergency equipment, maintenance requirements, training programs and operations manuals.

1. Complex Operations

This category would establish rules for operating around built-up areas and close to aerodromes. Because of the increased risk associated with such operations, this category would establish the most stringent requirements.

Specific requirements include:

• Aircraft marking and registration;
• Pilot permit (criteria includes: age restrictions, medical fitness, knowledge requirements, passing a Transport Canada written exam, practical training on the type of UAV to be flown and demonstrating competency in performing normal and emergency procedures); and
• Manufacturer declaration that the UAV meets a design standard

Transport Canada proposes to establish operating rules for this category including:

• Not operating within Class A or B airspace;
• Not operating within Class F Restricted airspace without permission;
• Restricting a single pilot from operating more than one UAV at the same time;
• Prohibiting visual observers from being situated in moving vehicles;
• Maintaining a minimum lateral distance from people, animals, buildings and vehicles (specific distances have not been proposed); and
• Operating at a maximum altitude of 400 feet AGL

2. Limited Operations

This category would apply to operations taking place in remote areas, for example, agricultural operations and rural aerial surveys. This category would mandate specific operating distances from built-up areas and aerodromes.

Specific requirements include:

• Aircraft marking and registration;
• Basic knowledge test; and
• Manufacturer declaration that the UAV meets a design standard

NOT required:

• Pilot permit

The operating rules for this category would include:

• Operating only during the daytime;
• Maintaining a maximum speed of 87 knots;
• Not entering in Class C, D, E or F airspace;
• Staying a certain distance away from aerodromes (proposal considers 9km and 20 km);
• Staying at least 9 km from built-up areas; and
• Operating at a maximum altitude of 300 feet AGL

3. Operations with Very Small UAVs

Transport Canada is seeking comments on whether it should incorporate a category that would establish less restrictive rules for very small UAVs, and whether the category should be based on weight or an alternative approach i.e. kinetic energy. Although the proposal does not specify the weight limitation that would be applied to this category, it references the 2kg exemption that was issued last November.

As an alternative for classifying very small UAVs, Transport Canada proposes using a table that assesses maximum weight and airspeed to determine the lethality. Another option considered is to create a low energy category for UAVs that would not impart energy over a certain threshold on impact.

Specific requirements include:

• Aircraft identification (pilot name and contact marked on UAV); and
• Basic knowledge test

NOT required:

• Aircraft marking and registration;
• Pilot permit;
• Medical certificate; and
• Manufacturer declaration that the UAV meets a design standard

The proposed operating and flight rules for this category include:

• Operating only during the daytime;
• Not entering in Class C, D, E or F airspace;
• Staying at least 9 km from aerodromes; and
• Operating at a maximum altitude of 300 feet AGL

If you would like to submit a comment to Transport Canada, you may send your comment to carrac@tc.gc.ca by August 28^th.

image: Sony Pictures

Chappie, the new robo-film on the block, takes place in Johannesburg, where the police force is made up of robots. In one of the early scenes, the main characters, Die Antwoord’s Ninja and Yolandi take part in a drug deal that is raided by robot cops. Hoping to avoid a similar fate in their next deal, Yolandi suggests that they find the robots’ remote so that they can switch them off like TV sets. Do the robots have a remote? And do Ninja and Yolandi find it? No spoilers here, but let’s take up the underlying question in the context of drone regulations…

What is Canada’s position on autonomous and automated operations? Does your drone have a ‘remote’? If it does, according to Transport Canada, it is not an autonomous drone. Transport Canada defines an autonomous drone as one that does “not allow pilot intervention in the management of the flight.” It is not enough for an autonomous drone to be capable of self-governance, rather it must not allow for any possibility of human intervention.

What about drones that have a remote but can complete automated tasks such as take-offs or landings or that can execute pre-defined waypoint operations? Transport Canada distinguishes these drones from autonomous drones by pointing to the fact that they require operator initiation or intervention.

Although there is no express prohibition, truly autonomous operations are outside of the scope of Canada’s current regulations. For the time being, if your drone doesn’t have a remote, you wouldn’t be able to operate. Our framework permits operations that involve automation, but it requires that an operator have the capability to intervene – or rather, as Ninja and Yolandi would hope – your flying robot must have a remote.

JJRC H8C Quadcopter Drone – Night Flight. Source YouTube.

The FAA’s Notice of Proposed Rulemaking (NPRM) for small unmanned aircraft systems (SUAS) limits SUAS operations to daylight hours. The FAA concedes that the restriction on nighttime flights may negatively impact the ability to use SUAS in northern regions such as Alaska that have very few hours of daylight during winter months. The FAA has invited stakeholders to comment on how risk may be mitigated in the course of nighttime operations. Here is a breakdown of the Canadian approach to risk management for nighttime flights…

In Canada, nighttime operations are possible under the Special Flight Operations Certificate (SFOC) system. Transport Canada’s Staff Instruction 623-001 outlines the following conditions for nighttime operations:

• If the UAV has lights, the lights must be on during night flights
• If the UAV does not have lights, there must be a means of illumination that would enable visual contact with the UAV
• The crew must have a portable emergency light source available
• The pilot must not have visual limitation of depth perception, colour blindness or problems seeing at night
• Visual observers providing the sense and avoid function (i.e. where the pilot is using FPV) cannot use night vision goggles

Transport Canada further states that if light emitting diodes (LEDs) are used to satisfy the lighting requirement for nighttime operations, the LEDs must have sufficient intensity to enable the pilot or visual observer and other airspace users to have visual contact with the UAV.

Source: Matt Chamberlain.

The FAA’s Notice of Proposed Rulemaking (NPRM) for small unmanned aircraft systems (SUAS) proposes to restrict operations that are completed beyond visual line-of-sight (BVLOS). However, the FAA has invited comments regarding the proposed BVLOS restriction. In Canada, BVLOS operations are not yet mainstream, however they are attainable under the Special Flight Operations Certificate (SFOC) system.

Transport Canada’s Staff Instruction 623-001 establishes the following conditions for BVLOS operations:

• BVLOS flights cannot take place outside of restricted airspace, unless the operator can mitigate risk to an acceptable level i.e. through the use of ground-based radar
• BVLOS flights must not be conducted over populated areas
• BVLOS flights must be conducted in visual meteorological conditions
• BVLOS flights cannot be conducted within controlled airspace
• BVLOS flights can only be conducted within 5 nautical miles of the point of departure
• The take-off and landing/recovery must be conducted within visual line-of-sight
• Direct radio line-of-sight capability must be maintained throughout the operating area

Canada is not alone in having established conditions for BVLOS operations. In fact, quite a few jurisdictions have more advanced regulations for BVLOS operations, including Australia, Colombia, Czech Republic, France, Israel and Poland. As stakeholders submit comments to the FAA over the next two months, it’s important to look at how other jurisdictions are managing risk in BVLOS operations.

Fotokite tethered drone.

In November, Transport Canada issued Staff Instruction 623-001, which provides guidance on the review and processing of Special Flight Operations Certificate (SFOC) applications. Among the issues addressed are whether an SFOC is required for indoor and tethered drone operations. Below is a summary of Transport Canada’s position…

Indoor Operations

According to the Staff Instruction, there are certain situations in which you don’t need an SFOC to conduct an operation inside a building or a structure, however this is not always the case. Whether you need an SFOC or not for an indoor operation turns on who is actually present while the operation is being conducted.

If only the operation crew is present, an SFOC is not required. Similarly, if only the crew and other people who are directly participating in the operation are present, an SFOC is not needed. For example, for an indoor movie set, an SFOC would not be required if only the crew and actors are present. However, if there are individuals present within the building or structure who are not directly involved in the operation, then an SFOC would be required. An example of this would be a sporting event where members of the public would be present.

Tethered Drones

The Staff Instruction characterizes tethered drones as “obstacles to air navigation [that] are to be marked and lit in accordance with the obstruction marking and lighting standards” found in the Canadian Aviation Regulations. The Staff Instruction further provides that tethered drones that are “extremely manoeuvrable and which operate over wide vertical/horizontal areas may require an SFOC.”

It is unclear what Transport Canada considers to be an ‘extremely manoeuvrable’ tethered drone or what constitutes a ‘wide vertical/horizontal area’. However, the Staff Instruction states that, “operating an aircraft on a tether simply to avoid SFOC requirements is not a viable solution.”

Source: Wikipedia

Canada’s relatively favorable framework for commercial UAV operations is attracting interest among foreign operators who are frustrated with restrictions in their home jurisdictions. The following provides a breakdown of Canadian regulations affecting foreign commercial operators…

Are foreign operators eligible to operate in Canada?

Foreign operators may conduct commercial operations in Canada if they are granted a Special Flight Operations Certificate (SFOC) from Transport Canada. To be considered a candidate for an SFOC to conduct a commercial operation, a foreign operator must demonstrate legal eligibility to operate in the operator’s home jurisdiction. For example, a US operator would require a Certificate of Waiver and Authorization (COA) or a Special Airworthiness Certificate (SAC) to be eligible to apply for an SFOC to conduct a commercial operation in Canada.

What options are available for foreign operators who aren’t legally eligible to operate in their home jurisdictions?

First, foreign operators who have no legal basis for operating in their home jurisdictions are permitted to conduct research and development operations in Canada under an SFOC. Such activities must be conducted at designated UAV test sites such as the CCUVS site recently approved in Southern Alberta. The CCUVS site operates under restricted airspace that spans 700 square nautical miles running up to 18,000 feet above sea level. Because the airspace is restricted, it is possible to complete more risky operations such as those conducted beyond visual line of sight.

Second, foreign operators who have no legal basis to operate in their home jurisdictions can set up a Canadian subsidiary through which they may conduct commercial operations. By establishing a Canadian corporate entity, operators can avoid the foreign eligibility requirements and apply through the normal SFOC process open to Canadian operators. For example, if a US corporation sets up a Canadian subsidiary, the Canadian entity can apply for an SFOC without having a COA or a SAC. Additionally, the Canadian entity would be eligible for the two new exemptions from the SFOC process (assuming the other exemption criteria are met).

If you liked this article, you may also be interested in:

• Transport Canada releases new framework for governing UAV operations
• Transport Canada relaxes commercial drone regulations, increases focus on academic use
• Commercial drone regulations: Canada vs. US

• Study shows most Americans are wary of commercial and personal drones
• Commercial drone use legal in USA? Who’s willing to make bail?
• Does the Small UAV industry need its own coalition?
• Snapshot of US UAV situation

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

When do you need an SFOC?

An SFOC is required if you are operating a UAV that weighs more than 35kg for any purpose. Operators must also obtain an SFOC if they are flying for “work or research” purposes and they do not meet the requirements of either of the new exemptions (for instance, if the UAV weighs more than 25kg). Before outlining the exemptions, it is important to highlight that the “research” criteria reflects an expansion of the general SFOC requirement.

When can you avoid an SFOC?

If a UAV is not being used for “work or research” and it weighs 35kg or less, an SFOC is not required, however operators are still expected to engage in safe practices. Transport Canada has enumerated “safety tips” such as: flying during daylight, within sight; avoiding airports, populated areas and moving vehicles; and not exceeding an altitude of 90 meters.

Those operating UAVs for “work or research” may be able to benefit from two exemptions.  The first of these applies to operators flying UAVs weighing less than 2kg. The requirements that the operator must satisfy to qualify for this exemption include: age restrictions, carrying liability insurance, flying during daylight in direct line of sight, and flying at a distance of at least 30 metres from people, animals, buildings and vehicles not involved in the operation.

The second exemption applies to operators flying UAVs weighing between 2kg and 25kg for “work or research” purposes. This exemption features a more stringent spin on the requirements that apply to UAVs under 2kg (i.e. staying at least 150 metres away from people, animals, buildings and vehicles not involved in the operation). Additional criteria includes: developing and adhering to landing and recovery procedures and having a fire extinguisher on site.

Reflecting on the new exemptions, Brendan Schulman, a New York attorney representing commercial operators commented that “Canada’s new regulatory framework reflects a thoughtful risk-based approach and recognizes that at low weights and low altitudes, commercial drones do not pose serious safety risks. I hope our regulators in the United States take note of this alternative path to the future regulation of commercial drones.”

First, an SFOC will no longer be required for commercial operators flying UAVs under 2 kilograms. And second, commercial operators conducting low risk operations using UAVs under 25kg will also be exempt from the SFOC requirement. Transport Canada has yet to release information regarding the factors that will determine whether an operation falls into the parameters of the second exemption.

A closer look at the Transport Canada announcement suggests additional implications are on the horizon. Here is the relevant statement:

Until the new requirements come into effect, you must apply for a Special Flight Operations Certificate if:

1. Your aircraft weighs more than 35 kilograms (77 pounds).
2. You use your aircraft for work or academic purposes (such as aerial photography, geomatic surveying, crop observation, advertising, research and development).

The academic purposes concept seems to be a new addition to the SFOC requirement. Let’s trace the evolution of the terminology employed in determining whether an SFOC is required…

Transport Canada enforces the Canadian Aviation Regulations (CARs) and the Aeronautics Act. Subsection 101.01(1) of the CARs defines a UAV as a “power-driven aircraft, other than a model aircraft, that is designed to fly without a human operator on board.” Section 603.66 prohibits flying a UAV without complying with SFOC requirements.

Under the CARs, model aircraft means an aircraft weighing no more than 35kg that is mechanically driven or launched for recreational purposes. The CARs do not define “recreational purposes”. However, Transport Canada’s Staff Instruction No. 623-001 points to the definition in the Aeronautics Act which turns on whether there is hire and reward, which is defined as “any payment, consideration, gratuity or benefit, directly or indirectly charged, demanded, received or collected by any person for the use of an aircraft”.

A few weeks ago, Transport Canada published an infographic on its website to assist UAV operators in determining whether they need to apply for an SFOC. The infographic states that if you are not using the UAV for “work” and it does not weigh more than 35kg, you do not need an SFOC – still no mention of academic purposes. It appears that the notice regarding the two upcoming exemptions might be the first instance of the academic purposes terminology.

What are the implications of the new terminology? Without a definition of “academic purposes”, it is unclear how far reaching the effects of the terminology will be. Is the term meant to capture research and development activities taking place in academic institutions? Will it apply to student projects outside of research lab environments? Transport Canada will need to explain what is meant by “academic purposes” to ensure that UAV operators have clarity regarding the application of the SFOC requirements in academia.

Source: Wikipedia

When Canadians attempt to characterize aspects of Canadian culture, it’s not uncommon to draw comparisons with the US. I recently noticed that as I respond to questions about the Canadian regulations surrounding commercial drones, I often begin by stating that our regulatory framework is quite distinct from that of the US – here’s why…

In Canada, commercial operators can apply to obtain Special Flight Operations Certificates (SFOCs) from Transport Canada. It takes Transport Canada about 20 days to assess applications, and last year the agency issued 945 SFOCs to applicants representing a variety of industries including aerial videography, agriculture and oil and gas.

Generally, the Canadian regulations do not establish bright line rules governing drone operations – for instance they do not specify whether you need a pilot’s license to complete a commercial drone flight, or whether it is permitted to fly beyond the visual line of sight. Rather, Transport Canada assesses applications using a case-by-case approach. In order to obtain approval, applicants must show that they can mitigate operational risks to an acceptable level.

In the US, the Federal Aviation Administration (FAA) has been working to develop drone regulations since the enactment of the FAA Modernization Act of 2012. Until the framework is in place, those looking to fly for commercial purposes can only proceed by exemption. Most companies have been denied exemptions, the notable exceptions being a couple of oil companies that received approval to operate drones in remote areas of Alaska.

Last Thursday, the FAA extended regulatory exemptions to six Hollywood companies looking to film using drones. Although the Hollywood exemptions represent a move in a positive direction, the restrictions placed on the companies are quite onerous, for instance the operations must take place in a controlled closed-set environment and may only be completed below 400 feet and within the visual line of sight.

By comparison, commercial drone operations are the norm in Canada and will continue to be an exception in the US until the new rules are in place.

The information contained herein should not be regarded as legal advice.

by Ryan Gariepy

I recently spent over a week in Montreal talking with – and learning from – the Canadian field robotics community, first at the Computer and Robot Vision (CRV) conference and then at the NCFRN Field Trials. This was my first time attending CRV since I spoke at the 2012 conference in Toronto, and it’s quite inspiring how much robotics is spreading throughout Canada. Keynote speakers were Raff D’Andrea from ETH-Z and Frank Dellaert from Georgia Tech, both of whom had excellent talks. Bonus points to Frank for pointing out factor graphs to me for the first time, and Raff for bringing the best tabletop demo ever!

CRV transitioned right into the opening meetings of the NCFRN Field Trials. Or, for those of you who don’t like nested acronyms, the National Science and Engineering Research Council Canadian Field Robotics Network Field Trials. The NCFRN makes up Canada’s leading field robotics researchers, companies, and supporting government organizations, and has just marked its second year in existence. The first two days are filled with presentations, seminars, and poster sessions, and from there it’s into the eponymous field. Something which was both surprising and inspiring for me was seeing that Clearpath hardware represented every single ground vehicle and surface vessel in the poster session.

Spread from the Mars Emulation Terrain at the Canadian Space Agency to the east (aka, the “Mars Yard”) to the McGill MacDonald campus to the west, the field trials were quite the experience. Researchers and partners were able to see the algorithms operating live. They were able to collaborate on combining their technologies, and, in the most extreme cases, swapped code between robots to test how robust their approaches were. In my opinion, one of the most valuable aspects of the field trials is improving how testing and deploying autonomous vehicles in remote locations is completed. Teams’ preparations ranged from being as minimal and lightweight as possible to turning an entire cube van into a remote lab.

This years’ field trials also mark an important milestone for Clearpath Robotics: It’s the first time we’ve seen someone make coffee using our robots (at least, the first time we’re aware of it). Apparently it’s easier to plug a coffee maker into a Grizzly than it is to run an extension cord. I can understand why they were making coffee; this was the ASRL team which did 26 repeated traverses of nearly a kilometer each based on a single, visually taught route. And, since the ASRL has a thing for setting videos of our robots to music, here’s the research set to music.

I’ll close off by sharing the kind of experience which keeps us going: On the last day of the trials, I could only see six robots left rolling around the Mars Yard. There were three kinds, and all were robots designed and built by the Clearpath team – we’re thrilled to see that our work is having an impact on the robotics community and we can’t wait to see how Year 3 Trials will go next spring!