By Anthony King

What if training a robot to handle dirty, dangerous work on the factory floor was as simple as showing it how? Czech startup RoboTwin is doing exactly that, helping factory workers teach robots new skills by demonstration.

Instead of writing complex code, workers perform the job once and RoboTwin’s technology turns those movements into a robot programme – opening the door to automation for smaller manufacturers.

Founded in Prague in 2021, RoboTwin builds handheld devices and no-code software that capture human movements and translate them into instructions for industrial robots. The aim is to make automation faster, simpler and more accessible to manufacturers that do not have specialist robotics programmers.

“The robot basically copies the human demonstration,” said Megi Mejdrechová, RoboTwin’s co-founder and chief technology officer. “People with no coding skills can transfer their know-how and experience to robots.”

Mejdrechová, a mechanical engineer trained at the Czech Technical University in Prague, developed the core technology behind RoboTwin during her work in robotics research and industry. Her experience in robot control using AI and computer vision inspired her to create something practical for European manufacturers.

“Czech engineering is quite traditional and focused on scientific papers,” said Mejdrechová. “Visits to Singapore and Canada and other work experiences led me to focus on making a product that people could use.”

Getting started

In 2021, Mejdrechová entered a jump‑starter programme and won first prize in the manufacturing category. “We saw then that there was potential for the technology,” she said.

This encouraged her to start RoboTwin with colleagues Ladislav Dvořák and David Polák, who shared her enthusiasm for human‑robot partnerships. Mejdrechová received backing from Women TechEU, an EU scheme supporting women founders of deep‑tech startups. 

The RoboTwin team shared their results on the Horizon Results Platform, an online showcase for EU‑funded innovations, which led to an invitation to the EU’s Empowering Start‑ups and SMEs initiative. 

This helped fund their trip to Hannover Messe 2025, a major global manufacturing trade fair, and opened doors to new business contacts and deals.

Through a mix of public and private investment, RoboTwin has secured funding to refine its technology and expand to manufacturers in Central Europe, the Netherlands, Mexico and Canada.

In 2025, Mejdrechová was named in Forbes Czechia’s 30 Under 30 list for her work in making the training of robots accessible to more manufacturers.

Schooling robots

At the heart of RoboTwin’s system is a handheld device equipped with sensors. When a worker performs a task, for example spray painting a metal component, the system records the movement and converts it into a robot programme that can be reused in production.

Instead of requiring a specialist engineer to manually code every movement, the system captures the worker’s natural technique and translates it into precise instructions a robot can follow.

“We started with jobs that are ugly, dirty and unhealthy for workers to do manually,” said Mejdrechová.

Thanks to the no‑coding system, the process can be completed in a few steps and typically takes about a minute. For factories producing small batches or frequently changing products, this speed can make automation far more practical than traditional robot programming.

Making automation easy for all

Robotics in manufacturing is not new. The automotive industry already leads the way with about 23 000 new robots added to production lines in 2024. But while large companies can invest heavily, automation remains challenging and expensive for many SMEs.

This is where RoboTwin lends a hand. It has assisted firms in the surface‑treatment industry – companies that powder coat, paint or polish metal or plastic parts for car factories.

“Even if the batch of products you are producing is small, with our approach you can create a robot programme fast and easily,” said Mejdrechová.

For example, RoboTwin has assisted RobPainting, a Dutch company that robotises painting for SMEs to improve quality, reduce costs and minimise rework. 

“With our device we can teach the robot precise trajectories that are needed for a product and about its surroundings,” said David Vobr, a robotics specialist at RoboTwin who often assists customers.

Dangerous jobs

RoboTwin’s system can work with a wide range of industrial robots, including collaborative robots designed to operate safely alongside humans.

“We can have manipulators or painting robots and also collaborative robots, which can work alongside humans because they have sensors that tell them when to stop moving if someone could get hurt,” said Vobr.

RoboTwin initially focused on surface treatment in manufacturing, where tasks such as spray painting require workers to wear protective clothing and perform repetitive movements.

“These jobs are difficult to automate because there is often a lot of hidden movement involved,” said Mejdrechová, referring to small adjustments and gestures that workers make instinctively.

The sector also faces labour shortages.

“People are often not happy doing these things and there is a lack of workers willing to take these jobs. So there is a high demand for automation.”

Customers report that many robot programmes can now be created without shutting down a production line.

RoboTwin has already worked with a number of companies, including Surfin Technology, a Czech company specialising in robotic coating solutions, and Innovative Finishing Solutions in Canada, which brings its technology to North American customers.

Scaling up

EU support for RoboTwin is ongoing. A €2.3 million grant from the European Innovation Council secured in 2025 will help accelerate product development and market expansion.

The funding will support the next generation of RoboTwin’s technology. Instead of relying solely on manual demonstrations, the system will increasingly use stored experience and data to generate robot programmes automatically based on the shape of an object.

The company says this could make automation viable for many manufacturing tasks that were previously too complex or costly to automate.

For Europe, technologies like RoboTwin could play an important role in strengthening digital sovereignty and smart industrial innovation. They can help smaller manufacturers adopt advanced robotics without needing specialised programming expertise.

As factories become more flexible and data-driven, the ability to quickly teach robots new tasks may prove increasingly valuable.

Mejdrechová believes this shift will help bring automation within reach of a much wider range of companies.

“Our goal is to make robot training something that workers can do themselves,” she said. “If we succeed, automation will no longer be limited to large factories with specialised engineers. It will become a tool that any manufacturer can use.”

Research in this article was funded by the EU’s Horizon Programme. The views of the interviewees don’t necessarily reflect those of the European Commission. If you liked this article, please consider sharing it on social media.

This article was originally published in Horizon, the EU Research and Innovation magazine.

By Anthony King

Modern surgery has gone from long incisions to tiny cuts guided by robots and AI. In the process, however, surgeons have lost something vital: the chance to feel inside the body directly. Without palpation, it becomes harder to detect tissue abnormalities during an operation.

A group of surgeons and engineers across Europe is now trying to bring back this vital aspect of surgery.

Working within an EU-funded research collaboration called PALPABLE, they are developing a soft robotic “fingertip” that can sense how firm or soft tissue is during minimally invasive and robotic surgery. The research runs until the end of 2026, with a first prototype expected to be tested by surgeons around March 2026.

By combining optical sensing, soft robotics and AI, the team is designing a probe that mimics the way a fingertip presses and feels during surgery. It would gently probe organs and create a visual map of tissue stiffness, displayed on a screen to guide surgeons as they operate.

Losing the surgeon’s touch

For many surgeons, the loss of direct touch has been one of the quiet trade-offs of modern surgery.

Sooner or later, I believe the vast majority of surgeries will be robotic.

“We started 30 years ago with open surgery and using our fingers,” said Professor Alberto Arezzo from the University of Turin, Italy. He specialises in minimally invasive and robotic surgery and mostly treats patients with colorectal cancer.

“Then we moved into the era of keyhole surgery, which reduced tactile feedback because we began to use long instruments,” he said.

From the 1990s, keyhole surgery became increasingly common, allowing surgeons to operate through small incisions with the help of a camera. Patients benefited from less trauma, shorter hospital stays and faster recovery.

But this came at the expense of physical touch. That matters because tumours often feel different from healthy tissue – stiffer, less pliable or irregular – important differences that experienced hands can detect.

Finding tumour margins

When operating on cancer, surgeons walk a fine line: remove too much tissue and function may suffer; remove too little and cancer may remain, and then spread again, requiring more surgery. 

“We don’t want to do that. We want it done in one shot,” said Dr Gadi Marom at Hadassah Medical Centre in Jerusalem, one of the clinicians involved in the research, who specialises in minimally invasive and robotic surgery on patients with stomach and oesophagus diseases. 

This is where sensing technology could help. By translating physical contact into visual information, such as a colour-coded map showing softer and firmer areas, surgeons could regain a functional equivalent of touch. 

“With a new instrument, we want to be able to determine the margins around a tumour,” said Marom.

Using light to feel

To do that, engineers on the team are turning to light.

The probe they are developing contains fibre-optic cables embedded in a soft, flexible tip. When pressed against tissue, the tip deforms and the light travelling through the fibres changes.

“A silicone dome presses against soft tissue, allowing us to map both the direction and the magnitude of the applied force,” explained Dr Georgios Violakis at Hellenic Mediterranean University in Heraklion, Crete. 

Those tiny shifts in light intensity and wavelength are then translated into information about tissue stiffness.

In the lab, the team has already built and calibrated early versions of the soft membrane and light-based sensors, with partners contributing across the system.

Queen Mary University of London (UK) is helping design and refine the membranes, the Fraunhofer Institute (Germany) is developing the functional films, while Bendabl (Greece), Tech Hive Labs (Greece) and the University of Essex (UK) are advancing the software needed to visualise stiffness and tactile maps.

The prototype will be validated in lab tests before it is used on patients.

The bottom line is that we will be able to give better care to our patients.

The fibre‑optic cables are each about the width of a human hair. Similar sensing technology has long been used to detect small movements in large structures such as aircraft, skyscrapers and nuclear reactors. Here it is being applied on a much smaller scale to detect subtle differences in human tissue. 

“For touching organs inside an anaesthetised patient, the device needs to be both highly accurate and high resolution,” said Professor Panagiotis Polygerinos, a soft robotics researcher at Hellenic Mediterranean University. 

“Something like this might have been possible sooner, but the technology would have been far more expensive and less precise, making it impractical for clinical use.” 

Bringing touch to robots

As surgery grows increasingly robotic, the loss of tactile feedback is becoming more pressing – and restoring a sense of touch even more vital.

“When I operate with a robot I have the advantage of 3D vision,” said Marom. “And I don’t have to stand for the entire surgery.” That matters in long procedures, such as removing a patient’s oesophagus, which can take up to eight hours.

Robotic surgery also raises new possibilities. Marom hopes it may eventually allow surgeons, in carefully selected cases, to remove small tumours from the oesophagus without removing the entire organ.

But there is a downside.

“In robotic surgery, tactile feedback is largely absent,” said Arezzo. “That’s why this work is so important.”

Both surgeons believe robotics will continue to expand in operating theatres, but only if surgeons are given better sensory information.

“Sooner or later, I believe the vast majority of surgeries will be robotic,” said Arezzo.

For Marom, working closely with engineers has been essential. “I am exposed to soft robotics and many new technologies,” he said. “I see how new instruments can be developed.”

“The bottom line is that we will be able to give better care to our patients,” he added.

Research in this article was funded by the EU’s Horizon Programme. The views of the interviewees don’t necessarily reflect those of the European Commission. If you liked this article, please consider sharing it on social media.

This article was originally published in Horizon, the EU Research and Innovation magazine.

By Michael Allen

In the critical 72 hours after an earthquake or explosion, a race against the clock begins to find survivors. After that window, the chances of survival drop sharply.

When a powerful earthquake hit central Italy on 24 August 2016, killing 299 people, over 5 000 emergency workers were mobilised in search and rescue efforts that saved dozens from the rubble in the immediate aftermath.

The pressure to move fast can create risks for first responders, who often face unstable environments with little information about the dangers ahead. But this type of rescue work could soon become safer and more efficient thanks to a joint effort by EU and Japanese researchers.

Supporting first responders

Rescue organisations, research institutes and companies from both Europe and Japan worked together from 2019 to 2023 to develop a new generation of tools blending robotics, drone technology and chemical sensing to transform how emergency teams operate in disaster zones.

It is a prototype technology that did not exist before.
– Tiina Ristmäe, CURSOR

Their work was part of a four-year EU-funded international research initiative called CURSOR, which included partners from six EU countries, Norway and the UK. It also included Tohoku University, whose involvement was funded by the Japan Science and Technology Agency.

The researchers hope that the sophisticated rescue kit they have developed will help rescue workers locate trapped survivors faster, while also improving their own safety.

“In the field of search and rescue, we don’t have many technologies that support first responders, and the technologies that we do have, have a lot of limitations,” said Tiina Ristmäe, a research coordinator at the German Federal Agency for Technical Relief and vice president of the International Forum to Advance First Responder Innovation.

Meet the rescue bots

At the heart of the researcher’s work is a small robot called Soft Miniaturised Underground Robotic Finder (SMURF). The robot is designed to navigate through collapsed buildings and rubble piles to locate people who may be trapped underneath.

The idea is to allow rescue teams to do more of their work remotely, localising and finding humans from the most hazardous areas in the early stages of a rescue operation. The SMURF can be remotely controlled by operators who stay at a safe distance from the rubble.

“It is a prototype technology that did not exist before,” said Ristmäe. “We don’t send people, we send machines – robots – to do the often very dangerous job.”

The SMURF is compact and lightweight, with a two-wheel design that allows it to manoeuvre over debris and climb small obstacles.

“It moves and drops deep into the debris to find victims, with multiple robots covering the whole rubble pile,” said Professor Satoshi Tadokoro, a robotics expert at Tohoku University and one of the project’s lead scientists.

The development team tested many designs before settling on the final SMURF prototype.

“We investigated multiple options – multiple wheels or tracks, flying robots, jumping robots – but we concluded that this two-wheeled design is the most effective,” said Tadokoro.

Sniffing for survivors

The SMURF’s small “head” is packed with technology: video and thermal cameras, microphones and speakers for two-way communication, and a powerful chemical sensor known as the SNIFFER.

This sensor is capable of detecting substances that humans naturally emit, such as C02 and ammonia, and can even distinguish between living and deceased individuals.

Put to the test in real-world conditions, the SNIFFER has proved able to provide reliable information even when surrounded by competing stimuli, like smoke or rain.

According to the first responders who worked with the researchers, the information provided by the SNIFFER is highly valuable: it helps them to prioritise getting help to those who are still alive, said Ristmäe.

Drone delivery

To further improve the reach of the SMURF, the researchers also integrated drone support into the system. Customised drones are used to deliver the robots directly to the areas where they’re needed most – places that may be hard or dangerous to access on foot.

Ιt moves and drops deep into the debris to find victims, with multiple robots covering the whole rubble pile.
– Professor Satoshi Tadokoro, Tohoku University

“You can transport several robots at the same time and drop them in different locations,” said Ristmäe.

Alongside these delivery drones, the CURSOR team developed a fleet of aerial tools designed to survey and assess disaster zones. One of the drones, dubbed the “mothership,” acts as a flying communications hub, linking all the devices on the ground with the rescue team’s command centre.

Other drones carry ground-penetrating radar to detect victims buried beneath debris. Additional drones capture overlapping high-definition footage that can be stitched together into detailed 3D maps of the affected area, helping teams to visualise the layout and plan their operations more strategically.

Along with speeding up search operations, these steps should slash the time emergency workers spend in dangerous locations like collapsed buildings.

Testing in the field

The combined system has already undergone real-world testing, including large-scale field trials in Japan and across Europe.

One of the most comprehensive tests took place in November 2022 in Afidnes, Greece, where the full range of CURSOR technologies was used in a simulated disaster scenario.

Though not yet commercially available, the prototype rescue kit has sparked global interest.

“We’ve received hundreds of requests from people wanting to buy it,” said Ristmäe. “We have to explain it’s not deployable yet, but the demand is there.”

The CURSOR team hopes to secure more funding to further enhance the technology and eventually bring it to market, potentially transforming the future of disaster response.

Research in this article was funded by the EU’s Horizon Programme. The views of the interviewees don’t necessarily reflect those of the European Commission. If you liked this article, please consider sharing it on social media.

This article was originally published in Horizon, the EU Research and Innovation magazine.

By Kaja Šeruga

Just outside the historic German town of Goslar, a sprawling industrial complex receives an endless stream of discarded electronics. On arrival, this electronic waste is laboriously prepared for recycling. 

Electrocycling GmbH is one of the largest e-waste recycling facilities in Europe. Every year, it processes up to 80 000 tonnes of electronic waste, which comes in all shapes and forms.

Manual dismantling

Despite an impressive array of machinery, more than half of the site’s employees manually prepare the discarded items for recycling. They do this by sorting the incoming waste and removing batteries, which are a fire hazard and a major challenge in e-waste recycling.

“There are more and more devices, they are getting smaller, and they all contain lithium batteries, some of which are permanently installed, soldered or glued in place,” said Hannes Fröhlich, Electrocycling’s managing director. 

“It’s not a dream job, dismantling these appliances every day with hammers and pliers. I think we can do better.”

Some of these tedious tasks could be performed by robots. However, the problem is that every time there is a change in the product or the process, the hardware and software need to be restructured. This can be costly and time-consuming.

To address this issue, an EU-funded research initiative named ReconCycle has managed to automate the process by creating robots that can reconfigure themselves for different tasks. 

New territory for robotics

Researchers from Slovenia, Germany and Italy worked together on this issue at the Jožef Stefan Institute, Slovenia’s leading research facility, from 2020 to 2024.

The team developed adaptable AI-supported robots that are able to remove batteries from smoke detectors and radiator heat metres.

These two products can be found in most households and are replaced every five to eight years, creating large amounts of waste.

“The main challenge is that there are so many different versions of each device. Just think how many different remote controls there are,” said Dr Aleš Ude. He is head of the Department of Automatics, Biocybernetics and Robotics at the Jožef Stefan Institute and coordinates the ReconCycle research team. 

In industrial settings, robots are usually programmed for one specific task, repeating exactly the same series of movements in a predictable environment. 

Instead, the researchers set out to create a robot that can adapt to many different tasks, using state-of-the-art AI. 

“We wanted to expand robotics, introduce robots where there aren’t any yet,” Ude said.

A growing problem

Working with Electrocycling, Ude’s international research team created an adaptable robotic work cell. This is a workspace that consists of at least one robot, its tools and equipment, and its controller.

The novelty here is that this closed system autonomously adapts itself to various tasks, with the help of complex AI-driven software and modular hardware that can be quickly reconfigured. It also uses soft components like SoftHand, a human-like hand that can manipulate objects with great precision.

There are also safety features like collaborative robots and emergency stop buttons.

International collaboration was crucial in securing the right expertise, said Ude. 

“Robotics is very interdisciplinary, so it’s difficult to find the right partners in one country.” 

Thankfully, the new robots are arriving just at the right time, as the amount of e-waste produced every year continues to grow. Almost 5 million tonnes of e-waste are produced in the EU each year, amounting to about 11 kilograms per person. Less than 40% of that is recycled, the European Parliament has warned. 

Globally, around 62 million tonnes of e-waste were produced in 2022 alone, enough to fill 1.5 million 40-tonne trucks, according to UN data. Even more worryingly, the amount of e-waste is rising five times faster than the amount that is being recycled.

The EU is working to reduce e-waste through the Waste from Electrical and Electronic Equipment Directive, which sets the standards for collection and recycling. 

The work of Ude’s team is also aligned with the EU’s digital strategy, which encourages the use of AI in manufacturing to improve efficiency and help achieve climate neutrality by 2050.

Throwing away money

E-waste also has serious economic implications. An estimated €84 billion is lost each year when valuable metals like copper, iron and gold are discarded instead of being reused, according to the UN’s global e-waste monitor. 

At Electrocycling, 80% of the e-waste is recovered as raw materials, such as iron, zinc, gold, silver and palladium – some 35 materials in all.

“People need to understand that this is not just waste, but also raw materials that need to be recycled and kept in circulation, both for economic efficiency and a reduction of CO2,” said Fröhlich. 

New technology can make it even more efficient, and Fröhlich sees a lot of potential in it. 

“I was surprised by how far the technology and AI have already come,” he said. “They even recreated a human hand for the robot.”

Ude hopes to continue working with Electrocycling to improve e-waste solutions further. The hope is also that adaptable robots which can handle changing environments will have applications far beyond e-waste recycling. 

Given more time and development, these robots could even handle general housekeeping, or support carers in senior homes, said Ude. 

“Robotics could be of great help in such areas.”

This article was originally published in Horizon, the EU Research and Innovation magazine.

The world’s fourth industrial revolution is ushering in big shifts in the workplace. © demaerre, iStock.com

Professor Steven Dhondt has a reassurance of sorts for people in the EU worried about losing their jobs to automation: relax.

Dhondt, an expert in work and organisational change at the Catholic University Leuven in Belgium, has studied the impact of technology on jobs for the past four decades. Fresh from leading an EU research project on the issue, he stresses opportunities rather than threats.

Right vision

‘We need to develop new business practices and welfare support but, with the right vision, we shouldn’t see technology as a threat,’ Dhondt said. ‘Rather, we should use it to shape the future and create new jobs.’

The rapid and accelerating advance in digital technologies across the board is regarded as the world’s fourth industrial revolution, ushering in fundamental shifts in how people live and work.

If the first industrial revolution was powered by steam, the second by electricity and the third by electronics, the latest will be remembered for automation, robotics and artificial intelligence, or AI. It’s known as “Industry 4.0”.

‘Whether it was the Luddite movement in the 1800s through the introduction of automatic spinning machines in the wool industry or concerns about AI today, questions about technology’s impact on jobs really reflect wider ones about employment practices and the labour market,’ said Dhondt.

He is also a senior scientist at a Netherlands-based independent research organisation called TNO.

The EU project that Dhondt led explored how businesses and welfare systems could better adapt to support workers in the face of technological changes. The initiative, called Beyond4.0, began in January 2019 and wrapped up in June 2023.

While the emergence of self-driving cars and AI-assisted robots holds big potential for economic growth and social progress, they also sound alarm bells.

More than 70% of EU citizens fear that new technologies will “steal” people’s jobs, according to a 2019 analysis by the European Centre for the Development of Vocational Training.

Local successes

The Beyond4.0 researchers studied businesses across Europe that have taken proactive and practical steps to empower employees.

“We shouldn’t see technology as a threat – rather we should use it to shape the future and create new jobs.”

– Professor Steven Dhondt, BEYOND4.0

One example is a family-run Dutch glass company called Metaglas, which decided that staying competitive in the face of technological changes required investing more in its own workforce.

Metaglas offered workers greater openness with management and a louder voice on the company’s direction and product development.

The move, which the company named “MetaWay”, has helped it retain workers while turning a profit that is being reinvested in the workforce, according to Dhondt.

He said the example shows the importance in the business world of managers’ approach to the whole issue.

‘The technology can be an enabler, not a threat, but the decision about that lies with management in organisations,’ Dhondt said. ‘If management uses technology to downgrade the quality of jobs, then jobs are at risk. If management uses technology to enhance jobs, then you can see workers and organisations learn and improve.’

The Metaglas case has fed into a “knowledge bank” meant to inform business practices more broadly.

Dhondt also highlighted the importance of regions in Europe where businesses and job trainers join forces to support people.

BEYOND4.0 studied the case of the Finnish city of Oulu – once a leading outpost of mobile-phone giant Nokia. In the 2010s, the demise of Nokia’s handset business threatened Oulu with a “brain drain” as the company’s engineers were laid-off.

But collaboration among Nokia, local universities and policymakers helped grow new businesses including digital spin-offs and kept hundreds of engineers in the central Finnish region, once a trading centre for wood tar, timber and salmon.

Some Nokia engineers went to the local hospital to work on electronic healthcare services – “e-health” – while others moved to papermaker Stora Enso, according to Dhondt.

Nowadays there are more high-tech jobs in Oulu than during Nokia’s heyday. The BEYOND4.0 team held the area up as a successful “entrepreneurial ecosystem” that could help inform policies and practices elsewhere in Europe.

Income support

In cases where people were out of work, the project also looked to new forms of welfare support.

Dhondt’s Finnish colleagues examined the impact of a two-year trial in Finland of a “universal basic income” – or UBI – and used this to assess the feasibility of a different model called “participation income.”

In the UBI experiment, participants each received a monthly €560 sum, which was paid unconditionally. Although UBI is often touted as an answer to automation, BEYOND4.0’s evaluation of the Finnish trial was that it could weaken the principle of solidarity in society.

The project’s participation income approach requires recipients of financial support to undertake an activity deemed useful to society. This might include, for example, care for the elderly or for children.

While detailed aspects are still being worked out, the BEYOND4.0 team discussed participation income with the government of Finland and the Finnish parliament has put the idea on the agenda for debate.

Dhondt hopes the project’s findings, including on welfare support, will help other organisations better navigate the changing tech landscape.

Employment matchmakers

Another researcher keen to help people adapt to technological changes is Dr Aisling Tuite, a labour-market expert at the South East Technical University in Ireland.

“We wanted to develop a product that could be as useful for people looking for work as for those supporting them.”

– Dr Aisling Tuite, HECAT

Tuite has looked at how digital technologies can help job seekers find suitable work.

She coordinated an EU-funded project to help out-of-work people find jobs or develop new skills through a more open online system.

Called HECAT, the project ran from February 2020 through July 2023 and brought together researchers from Denmark, France, Ireland, Slovenia, Spain and Switzerland.

In recent years, many countries have brought in active labour-market policies that deploy computer-based systems to profile workers and help career counsellors target people most in need of help.

While this sounds highly targeted, Tuite said that in reality it often pushes people into employment that might be unsuitable for them and is creating job-retention troubles.

‘Our current employment systems often fail to get people to the right place – they just move people on,’ she said. ‘What people often need is individualised support or new training. We wanted to develop a product that could be as useful for people looking for work as for those supporting them.’

Ready to run

HECAT’s online system combines new vacancies with career counselling and current labour-market data.

The system was tested during the project and a beta version is now available via My Labour Market and can be used in all EU countries where data is available.

It can help people figure out where there are jobs and how to be best positioned to secure them, according to Tuite.

In addition to displaying openings by location and quality, the system offers detailed information about career opportunities and labour-market trends including the kinds of jobs on the rise in particular areas and the average time it takes to find a position in a specific sector.

Tuite said feedback from participants in the test was positive.

She recalled one young female job seeker saying it had made her more confident in exploring new career paths and another who said knowing how long the average “jobs wait” would be eased the stress of hunting.

Looking ahead, Tuite hopes the HECAT researchers can demonstrate the system in governmental employment-services organisations in numerous EU countries over the coming months. 

‘There is growing interest in this work from across public employment services in the EU and we’re excited,’ she said.

Research in this article was funded by the EU.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Virtual-reality technology could help cure people of phobias including about spiders. © Leena Robinson, Shutterstock.com

By Helen Massy-Beresford

Imagine a single technology that could help a robot perform safety checks at a nuclear plant, cure a person’s arachnophobia and simulate the feeling of a hug from a distant relative.

Welcome to the world of “extended reality”. Researchers funded by the EU have sought to demonstrate its enormous potential.

Relevant research

Their goal was to make augmented reality, in which the real world is digitally enhanced, and virtual reality – a fully computer-generated environment – more immersive for users.

One of the researchers, Erik Hernandez Jimenez, never imagined the immediate relevance of a project that he led when it started in mid-2019. Within a year, the Covid-19 pandemic had triggered countless lockdowns that left people working and socialising through video connections from home.   

‘We thought about how to apply this technology, how to feel human touch even at a distance, when we were all locked at home and contact with others was through a computer,’ said Hernandez Jimenez. 

He coordinated the EU research initiative, which was named TACTILITY and ran from July 2019 until the end of September 2022. 

The TACTILITY team developed a glove that simulates the sense of touch. Users have the sensation of touching virtual objects through electrical pulses delivered by electrodes embedded in the glove.

The sensations range from pushing a button and feeling pressure on the finger to handling a solid object and feeling its shape, dimensions and texture. 

Glove and suit

‘TACTILITY is about including tactile feedback in a virtual-reality scenario,’ said Hernandez Jimenez, who is a project manager at Spanish research institute TECNALIA.

He said the principle could be extended from the glove to a whole body suit. 

Compared with past attempts to simulate touch sensations with motors, the electro-tactile feedback technique produces a more realistic result at a lower cost, according to Hernandez Jimenez. 

This opens up the possibility of making the technology more widely accessible. 

The research bolsters European Commission efforts to develop the virtual-worlds domain, which could provide 860 000 new jobs in Europe this decade as the worldwide sector grows from €27 billion in 2022. 

The EU has around 3 700 companies, research organisations and governmental bodies that operate in this sphere, according to the Commission.

Phobias to factories

The TACTILITY researchers looked at potential healthcare applications. 

“We thought about how to apply this technology, how to feel human touch even at a distance.”

– Erik Hernandez Jimenez, TACTILITY

That’s where spiders come into the picture. They were among the objects in the project’s experiments to mimic touch.

‘One that was quite impressive – although I didn’t like it at all – was feeling a spider or a cockroach crawling over your hand,’ Hernandez Jimenez said.  

A potential use for the technology is treating phobias through exposure therapy in which patients are gradually desensitised to the source of their fear. That could start by virtually “touching” cartoon-like creepy crawlies before progressing to more lifelike versions.  

The tactile glove can also be used in the manufacturing industry, helping the likes of car manufacturers train their workers to perform tricky manoeuvres on the factory floor.

Furthermore, it can help people collaborate more effectively with remotely controlled robots in hazardous environments. An example is a nuclear power plant, where a person in a control room can virtually “feel” what a robot is touching. 

‘They get another sense and another kind of feedback, with more information to perform better checks,’ Hernandez Jimenez said. 

Joyful and playful

Wearables for virtual reality. © Oğuz ‘Oz’ Buruk, 2021

Wearable technologies for virtual-reality environments are also being inspired by the gaming industry. 

Researchers in a second EU-funded project sought to expand the prospects for technologies already widely used for professional purposes. The initiative, called WEARTUAL, ran from May 2019 until late 2021.

“Wearables are fashion items – they’re part of the way we construct our identity.”

– Oğuz ‘Oz’ Buruk, WEARTUAL

‘Our project focused on the more experiential side – joyful and playful activities,’ said Oğuz ‘Oz’ Buruk, who coordinated WEARTUAL and is assistant professor of gameful experience at Tampere University in Finland. 

Until recently, experiencing a virtual-reality environment involved a hand-held controller or head-mounted display. 

The WEARTUAL researchers looked at ways of incorporating wearables worn, for example, on the wrist or ankle into virtual reality to give people a sense of greater immersion. 

That could mean having their avatar – a representative icon or figure in the virtual world – blush when nervous or excited to enhance their ability to express themselves.

On the cusp

The team developed a prototype that could integrate varying physical sensations into the virtual world by transferring to it real-life data such as heart rate.  

Buruk is interested in how games will look in the “posthuman” era, when people and machines increasingly converge through bodily implants, robotics and direct communication between the human brain and computers.  

He signals that it’s hard to overestimate the eventual impact of advances in this area on everyday life, albeit over varying timescales: wearables are likely to be much more widely used in virtual reality in the next decade, while widespread use of bodily implants is more likely to take 50 to 100 years.

As technology and human bodies become ever more closely linked, the experience of transferring them to a virtual world will be enhanced, encouraging people to spend increasing amounts of time there, according to Buruk.

Virtual-reality technologies are already being used for practical purposes such as gamifying vital information including fire-safety procedures, making it more interactive and easier to learn. This type of use could expand to many areas.

On a very different front, several fashion houses already sell clothes that can be worn in virtual environments, allowing people to express their identity and creativity.  

‘Wearables are fashion items – they’re part of the way we construct our identity,’ Buruk said. ‘Investments in virtual reality, extended reality and augmented reality are increasing every day.’

Research in this article was funded by the EU via the Marie Skłodowska-Curie Actions (MSCA).

This article was originally published in Horizon, the EU Research and Innovation magazine.

Ocean divers could soon benefit from connected underwater technology. © Kirk Wester, Shutterstock.com

By Helen Massy-Beresford

Imagine seals swimming in the sea with electronic tags that send real-time water data to scientists back in their laboratories. Or archaeologists near a coast being automatically alerted when a diver trespasses on a precious shipwreck.

Such scenarios are becoming possible as a result of underwater connected technologies, which can help monitor and protect the world’s oceans. They can also shed light on the many remaining mysteries of the sea.

New frontier

‘A lot of funding has been provided to companies and institutions exploring space, but we have oceans around us that we have not explored,’ said Vladimir Djapic, innovation associate at the EU-funded TEUTA project.

“We have oceans around us that we have not explored.”

– Vladimir Djapic, TEUTA

Around 70% of the Earth is covered by oceans and more than four-fifths of them have never been mapped, explored or even seen by humans.

The Internet of Underwater Things, or IoUT, is a network of smart, interconnected sensors and devices to make communicating in the sea easier. It contrasts with the Internet of Things, or IoT, covering everything from smart phones to devices that allow people to switch on home heating remotely,

TEUTA ran from October 2020 through March 2022. It helped a Croatian company, H20 Robotics, develop and sell lightweight low-cost acoustic devices and robotic platforms for underwater wireless networks.

‘With a limited number of underwater network installations before, we could only explore limited coastal areas,’ said Djapic, who is chief executive officer of Zagreb-based H20 Robotics.

Advances in underwater technologies are expected to transform many sectors including marine biology, environmental monitoring, construction and geology.

Whale-like ways

TEUTA developed acoustic technology, which mimics the way whales and dolphins communicate.

Acoustic waves, unlike radio or optical communication ones, travel long distances underwater regardless of whether it is murky or clear.

Remote sensors, measuring tools, detection systems or cameras set up at an underwater site gather data then sent to a buoy on the surface. The buoy in turn sends the information wirelessly back to base, via the cloud, without the need for communication cables.

One focus area is improving communications between divers and land-based colleagues, according to Djapic.

‘For example, a diver working in underwater construction can send a message to a supervisor and request additional help or tools or similar,’ said Djapic.

Improved underwater communications will help connect land and sea, © H2O ROBOTICS, 2023

Scientists also stand to benefit by, for example, being able to remotely turn on a water-quality measuring device installed on the seabed from their labs.

For their part, archaeologists could use the technology to help protect vulnerable underwater sites with intruder-detection technology installed in remote locations.

Indeed, TEUTA technology will support another EU-backed project, TECTONIC, seeking to improve the documentation and protection of underwater cultural heritage at three pilot sites.

The sites are the Capo Rizzuto Marine Protected Area in southern Italy, the submerged ancient harbour of Aegina in Greece’s Saronic Gulf and a shipwreck site in the Deseado estuary in Argentina.

Other possibilities such as underwater agriculture or mining could also open up, according to Djapic.

For public agencies or non-governmental organisations that monitor water quality, the technology could replace the need for researchers to go and collect samples physically and deliver them to the lab.

While TEUTA gave a boost to fledgling underwater communication technologies, more work needs to be done in marketing them and ensuring they are used more widely, according to Djapic.

‘It all needs to be analysed,’ he said. ‘Our technology enables the measuring of environmental parameters.’

Sensors and samplers

Meanwhile, in Italy, a team of researchers is pursuing a new approach to ocean-data collection by using sensors and samplers that could be integrated into existing observatories and platforms.

This would enable the gathering of vast amounts of information useful for, as an example, the proposed European Digital Twin of the Ocean announced in February 2022. The twin will be a real-time digital replica of the ocean integrating both historical and live data.

By developing a new generation of marine technologies, the EU-funded NAUTILOS project will gather previously inaccessible information and improve understanding of physical, chemical and biological changes in oceans.

“They are the largest habitats on Earth, but the least observed.”

– Gabriele Pieri, NAUTILOS

Running for four years through September 2024, the project is coordinated by Gabriele Pieri of the Rome-based National Research Council.

‘Our proposal set out to fill a gap in the observation of oceans,’ said Pieri. ‘They are the largest habitats on Earth, but the least observed ones because of the difficulties in on-site observation and the costs of monitoring.’

NAUTILOS technology is already being tested in the Baltic and the Mediterranean seas, including the Aegean and Adriatic.

Sensors can, for example, measure levels of chlorophyll-A and dissolved oxygen in the water. These are important indicators of water quality and, by extension, of the presence of fish, helping protect their stocks.

Sensors and samplers collecting information about the concentration of microplastics in the water also expand understanding of the impact of human-generated pollution on the oceans.

Helping flippers and hands

One of the NAUTILOS partners, France’s National Centre for Scientific Research (CNRS), has even recruited some unlikely teammates: seals.

Swimming off the Valdes Peninsula in Argentina, these sea creatures have been tagged with sensors that record valuable data about the animals themselves and their habitats.

The NAUTILOS team, made up of research institutions and companies, is developing more than a dozen types of sensors and samplers. These include remote sensing technologies and microplastics detectors.

The project is keen to demonstrate that the new tools can work with existing and future platforms and easily switch between them.

The tools are relatively cheap, can be deployed quickly and work in conjunction with other equipment, offering many advantages. For example, a sensor can be mounted on an autonomous underwater vehicle and then moved to a fixed buoy.

Citizen science is an important part of NAUTILOS, which works with volunteers organising campaigns around ocean plastics, for example, as well as with scuba-diving associations whose members can test new technologies and offer feedback.

The team has also developed a smartphone app for divers to upload photos of underwater flora or fauna that can be assessed by researchers.

‘The interest in citizen science has really surprised me,’ said Pieri. ‘A lot of people are willing to help improve the life of the sea.’

Research in this article was funded by the EU and via the EU’s Marie Skłodowska-Curie Actions (MSCA). If you liked this article, please consider sharing it on social media.

This article was originally published in Horizon, the EU Research and Innovation magazine.

With technology for drones far advanced, the next step is to ensure they can fly safely in cities. Image credit: CC0 via Unsplash

The Spanish resort town of Benidorm is known for its sandy beaches with clear waters, a skyline dominated by towering hotels and tourists from northern Europe. But one day in February, it also served as a testing ground for European society’s future with drones.

Since the local economy depends on tourism during the summer, Benidorm is relatively empty in winter – and that’s a plus when it comes to safety while testing unmanned aerial vehicles (UAVs). The tall buildings that dominate the skyline also stand in nicely for those of a big city.

Sun, sea and…satellite signals

In sum, it’s an ideal place to try out new drone technology. And an EU-funded project called DELOREAN has done just that – testing new types of satellite tracking for drones on 9 February.

‘Benidorm’s skyline is quite similar to what you would find in larger cities like, say, New York,’ said Santiago Soley, the project coordinator who is also chief executive officer of Spanish aeronautics-engineering company Pildo Labs. ‘Generally, regulations limit drone flights over dense urban areas. It’s the first time in Europe we did these intense tests in a challenging city environment.’

Drones have been a hyped technology for years, during which the media popularised predictions that such aircraft would soon be used for all kinds of daily services including delivering packages to people’s doorsteps. Yet so far, widespread civilian use has failed to take off.

The bottleneck is safety and the need to demonstrate to city governments that drones can be operated in large numbers in populated areas without being a hazard. If a UAV crashes onto a busy street or into a plane that’s landing or taking off, the result could be severe damage or even deaths.

“Drone technology is getting there.”

– Santiago Soley, DELOREAN

Scientists and companies are now addressing these concerns – and the experiments in Benidorm might hold the key to the future success of drones.

‘Drone technology is getting there – it’s the least of our problems,’ said Soley. ‘What’s more important is to demonstrate how drones would safely be deployed over cities.’

DELOREAN is wrapping up after three years. The main goal was to develop navigation and positioning requirements for urban air services and show how the European Global Navigation Satellite System, or EGNSS, can help. 

Non-GPS options

Drones need to know exactly where they are at all times. For that, UAVs currently rely on satellites, mostly the US Global Positioning System, or GPS. Another alternative to GPS is Europe’s Galileo network.

DELOREAN is also testing Galileo’s potential for drones.

While led by Pildo Labs, the project has featured an international consortium whose members include France-based aircraft manufacturer Airbus, Spanish postal-servicer provider Correos and the European Organisation for the Safety of Air Navigation, or Eurocontrol, in Belgium. 

A challenge for satellite tracking in urban areas is that signals might be deflected or otherwise hindered by buildings. Galileo will help avoid such disruptions because of the waveform and structure of its signals, according to Soley.

In addition, Galileo is pioneering new services that could pinpoint drones’ locations with higher accuracy – something DELOREAN tested in Benidorm.

Furthermore, Galileo adds a layer of security. An authentication service that allows the drone to verify whether the satellite signal is real would counter any future efforts by criminal groups to misdirect UAVs and steal their contents through fake signals, according to Soley.

Airborne parcel deliveries

If experiments of the kinds conducted by DELOREAN prove successful, many applications could open up.

“Before businesses like urban air delivery can develop, we first need safety.”

– Professor Luis Moreno Lorente, LABYRINTH

While drones are already in use over cities, it is often in small-scale operations by local authorities. Police departments, for one, use them to monitor crowds or track speeding cars.

‘There are limitations on drone flights and you need to close the area,’ said Soley. ‘At the technical level, however, the flights are quite easy to handle.’

The next step could be mass urban air delivery. No more vans zigzagging through city streets with all the congestion and pollution.

Instead, fleets of drones would drop off packages across town. Companies like Amazon are already rolling out these services in limited areas.

‘Logistics will, I think, be one of the most promising uses of drones,’ said Soley.

Self-flying craft

An EU-funded project called LABYRINTH is tackling the challenge of ensuring that autonomous drones keep track of each other.

An ARQUIMEA drone being tested in Marugán, Segovia, Spain. © Labyrinth, 2023

Autonomous drones require no ground-based human pilots, who are generally needed for the current generation of UAVs. 

‘In the future, those drones will be operated autonomously – they will fly themselves,’ said Luis Moreno Lorente, the project coordinator and a professor of systems engineering and automation at the University Carlos III of Madrid in Spain. ‘But if you want to do that safely, you need to know exactly where each one of them is located.’

LABYRINTH, which is due to end in May after three years, is developing software that acts as an air traffic control system for drones. The 3D position of each is tracked and the aircraft then relays this information to other drones in the vicinity so they don’t crash into each other.

Similarly, if a drone faces technical troubles – say one of its motors fails – it needs to be able to direct other UAVs away from it.

‘Before businesses like urban air delivery can develop, we first need safety,’ said Moreno Lorente. ‘That’s what we’re building now.’

Together, LABYRINTH and DELOREAN are helping to clear the way for a future in which large numbers of drones fly over cities.

‘It’s just a matter of time before they do,’ said Moreno Lorente.

Watch the video

This article was originally published in Horizon, the EU Research and Innovation magazine.

Advanced robotics can help surgeons carry out procedures where there is little margin for error. © Microsure BV, 2022

In a surgery in India, a robot scans a patient’s knee to figure out how best to carry out a joint replacement. Meanwhile, in an operating room in the Netherlands, another robot is performing highly challenging microsurgery under the control of a doctor using joysticks.

Such scenarios look set to become more common. At present, some manual operations are so difficult they can be performed by only a small number of surgeons worldwide, while others are invasive and depend on a surgeon’s specific skill.

Advanced robotics are providing tools that have the potential to enable more surgeons to carry out such operations and do so with a higher rate of success.

‘We’re entering the next revolution in medicine,’ said Sophie Cahen, chief executive officer and co-founder of Ganymed Robotics in Paris.

New knees

Cahen leads the EU-funded Ganymed project, which is developing a compact robot to make joint-replacement operations more precise, less invasive and – by extension – safer.

The initial focus is on a type of surgery called total knee arthroplasty (TKA), though Ganymed is looking to expand to other joints including the shoulder, ankle and hip.

Ageing populations and lifestyle changes are accelerating demand for such surgery, according to Cahen. Interest in Ganymed’s robot has been expressed in many quarters, including distributors in emerging economies such as India.

‘Demand is super-high because arthroplasty is driven by the age and weight of patients, which is increasing all over the world,’ Cahen said.

Arm with eyes

Ganymed’s robot will aim to perform two main functions: contactless localisation of bones and collaboration with surgeons to support joint-replacement procedures.

It comprises an arm mounted with ‘eyes’, which use advanced computer-vision-driven intelligence to examine the exact position and orientation of a patient’s anatomical structure. This avoids the need to insert invasive rods and optical trackers into the body.

“We’re entering the next revolution in medicine.”

– Sophie Cahen, Ganymed

Surgeons can then perform operations using tools such as sagittal saws – used for orthopaedic procedures – in collaboration with the robotic arm.

The ‘eyes’ aid precision by providing so-called haptic feedback, which prevents the movement of instruments beyond predefined virtual boundaries. The robot also collects data that it can process in real time and use to hone procedures further.

Ganymed has already carried out a clinical study on 100 patients of the bone-localisation technology and Cahen said it achieved the desired precision.

‘We were extremely pleased with the results – they exceeded our expectations,’ she said.

Now the firm is performing studies on the TKA procedure, with hopes that the robot will be fully available commercially by the end of 2025 and become a mainstream tool used globally.

‘We want to make it affordable and accessible, so as to democratise access to quality care and surgery,’ said Cahen.

Microscopic matters

Robots are being explored not only for orthopaedics but also for highly complex surgery at the microscopic level.

The EU-funded MEETMUSA project has been further developing what it describes as the world’s first surgical robot for microsurgery certified under the EU’s ‘CE’ regulatory regime.

Called MUSA, the small, lightweight robot is attached to a platform equipped with arms able to hold and manipulate microsurgical instruments with a high degree of precision. The platform is suspended above the patient during an operation and is controlled by the surgeon through specially adapted joysticks.

In a 2020 study, surgeons reported using MUSA to treat breast-cancer-related lymphedema – a chronic condition that commonly occurs as a side effect of cancer treatment and is characterised by a swelling of body tissues as a result of a build-up of fluids.

MUSA’s robotic arms. Microsure BV, 2022

To carry out the surgery, the robot successfully sutured – or connected – tiny lymph vessels measuring 0.3 to 0.8 millimetre in diameter to nearby veins in the affected area.

‘Lymphatic vessels are below 1 mm in diameter, so it requires a lot of skill to do this,’ said Tom Konert, who leads MEETMUSA and is a clinical field specialist at robot-assisted medical technology company Microsure in Eindhoven, the Netherlands. ‘But with robots, you can more easily do it. So far, with regard to the clinical outcomes, we see really nice results.’

Steady hands

When such delicate operations are conducted manually, they are affected by slight shaking in the hands, even with highly skilled surgeons, according to Konert. With the robot, this problem can be avoided.

MUSA can also significantly scale down the surgeon’s general hand movements rather than simply repeating them one-to-one, allowing for even greater accuracy than with conventional surgery.

‘When a signal is created with the joystick, we have an algorithm that will filter out the tremor,’ said Konert. ‘It downscales the movement as well. This can be by a factor-10 or 20 difference and gives the surgeon a lot of precision.’

In addition to treating lymphedema, the current version of MUSA – the second, after a previous prototype – has been used for other procedures including nerve repair and soft-tissue reconstruction of the lower leg.

Next generation

Microsure is now developing a third version of the robot, MUSA-3, which Konert expects to become the first one available on a widespread commercial basis.

“When a signal is created with the joystick, we have an algorithm that will filter out the tremor.”

– Tom Konert, MEETMUSA

This new version will have various upgrades, such as better sensors to enhance precision and improved manoeuvrability of the robot’s arms. It will also be mounted on a cart with wheels rather than a fixed table to enable easy transport within and between operating theatres.

Furthermore, the robots will be used with exoscopes – a novel high-definition digital camera system. This will allow the surgeon to view a three-dimensional screen through goggles in order to perform ‘heads-up microsurgery’ rather than the less-comfortable process of looking through a microscope.

Konert is confident that MUSA-3 will be widely used across Europe and the US before a 2029 target date.

‘We are currently finalising product development and preparing for clinical trials of MUSA-3,’ he said. ‘These studies will start in 2024, with approvals and start of commercialisation scheduled for 2025 to 2026.’

MEETMUSA is also looking into the potential of artificial intelligence (AI) to further enhance robots. However, Konert believes that the aim of AI solutions may be to guide surgeons towards their goals and support them in excelling rather than achieving completely autonomous surgery.

‘I think the surgeon will always be there in the feedback loop, but these tools will definitely help the surgeon perform at the highest level in the future,’ he said.

Research in this article was funded via the EU’s European Innovation Council (EIC).

This article was originally published in Horizon, the EU Research and Innovation magazine.

Robotics and AI are poised to fundamentally change the future of healthcare. © Elnur, Shutterstock

In a Swiss classroom, two children are engrossed in navigating an intricate maze with the help of a small, rather cute, robot. The interaction is easy and playful – it is also providing researchers with valuable information on how children learn and the conditions in which information is most effectively absorbed.

Rapid improvements in intuitive human-machine interactions (HMI) are poised to kick off big changes in society. In particular, two European research projects give a sense of how these trends could influence two core areas: education and healthcare.

Child learning

In EU-funded ANIMATAS, a cross-border network of universities and industrial partners is exploring if, and how, robots and artificial intelligence (AI) can help us learn more effectively. One idea is around making mistakes: children can learn by spotting and correcting others’ errors – and having a robot make them might be useful.

‘A teacher can’t make mistakes,’ said project coordinator Professor Mohamed Chetouani of the Sorbonne University in Paris, France. ‘But a robot? They could. And mistakes are very useful in education.’

According to Prof Chetouani, it is simplistic to ask questions like ‘can robots help children learn better’ because learning is such a complex concept. He said that, for example, any automatic assumption that pupils who concentrate on lessons are learning more isn’t necessarily true.

That’s why, from the start, the project set out to ask smarter, more specific questions that would help identify just how robots could be useful in classrooms. 

ANIMATAS is made up of sub-projects each led by an early-stage researcher. One of the sub-project goals was to better understand the learning process in children and analyse what types of interaction best help them to retain information.

“Mistakes are very useful in education.”

– Professor Mohamed Chetouani, ANIMATAS

Robot roles

An experiment set up to investigate this question invited children to team up with the aptly named QTRobot to find the most efficient route around a map.

During the exercise, the robot reacts interactively with the children to offer tips and suggestions. It is also carefully measuring various indicators in the children’s body language such as eye contact and direction, tone of voice and facial expression.

As hoped, researchers did indeed find that certain patterns of interaction corresponded with improved learning. With this information, they will be better able to evaluate how well children are engaging with educational material and, in the longer term, develop strategies to maximise such engagement – thereby boosting learning potential.

Future steps will include looking at how to adapt this robot-enhanced learning to children with special educational needs.

‘We believe that it could be really important in this context,’ said Prof Chetouani.

Help at hand

Aki Härmä, a researcher at Philips Research Eindhoven in the Netherlands, believes that robotics and AI are going to fundamentally change healthcare.

“Healthcare can be 24/7.”

Aki Härmä, PhilHumans

In the EU-funded PhilHumans project that he is coordinating, early-stage researchers from five universities across Europe work with two commercial partners – R2M Solution in Spain and Philips Electronics in the Netherlands – to learn how innovative technologies can improve people’s health.

AI makes new services possible and ‘it means healthcare can be 24/7,’ Härmä said.

He points to the vast potential for technology to help people manage their own health from home: apps able to track a person’s mental and physical state and spot problems early on, chatbots that can give advice and propose diagnoses, and algorithms for robots to navigate safely around abodes.

Empathetic bots

The project, which started in 2019 and will run until late 2023, is made of up of eight sub-projects, each led by a doctoral student.

One sub-project, supervised by Phillips researcher Rim Helaoui, is looking at how the specific skills of mental-health practitioners – such as empathy and open-ended questioning – may be encoded into an AI-powered chatbot. This could mean that people with mental-health conditions would be able to access relevant support from home, potentially at a lower cost.

The team quickly realised that replicating the full range of psychotherapeutic skills in a chatbot would involve challenges that could not be solved all at once. It focused instead on one key challenge: how to generate a bot that displayed empathy.

‘This is the essential first step to get people to feel they can open up and share,’ said Helaoui.

As a starting point, the team produced an algorithm able to respond with the appropriate tone and content to convey empathy. The technology has yet to be converted into an app or product, but provides a building block that could be used in many different applications.

Rapid advances

PhilHumans is also exploring other possibilities for the application of AI in healthcare. An algorithm is being developed that can use ‘camera vision’ to understand the tasks that a person is trying to carry out and analyse the surrounding environment.

The ultimate goal would be to use this algorithm in a home-assistant robot to help people with cognitive decline complete everyday tasks successfully.

One thing that has helped the project overall, said Härmä, is the speed with which other organisations have been developing natural language processors with impressive capabilities, like GPT-3 from OpenAI. The project expects to be able to harness the unexpectedly rapid improvements in these and other areas to advance faster.

Both ANIMATAS and PhilHumans are actively working on expanding the limits of intuitive HMI.

In doing so, they have provided a valuable training ground for young researchers and given them important exposure to the commercial world. Overall, the two projects are ensuring that a new generation of highly skilled researchers is equipped to lead the way forward in HMI and its potential applications.

Research in this article was funded via the EU’s Marie Skłodowska-Curie Actions (MSCA).

This article was originally published in Horizon, the EU Research and Innovation magazine.

Robotics and AI can help build healthier bee colonies, benefitting biodiversity and food supply. © 0 Lorenzo Bernini 0, Shutterstock.com

The robotic bee replicants home in on the unsuspecting queen of a hive. But unlike the rebellious replicants in the 1982 sci-fi thriller Blade Runner, these ones are here to work.

Combining miniature robotics, artificial intelligence (AI) and machine learning, the plan is for the robotic bees to stimulate egg laying in the queen by, for example, feeding her the right foods at the right time.

Survive and thrive

‘We plan to affect a whole ecosystem by interacting with only one single animal, the queen,’ said Dr Farshad Arvin, a roboticist and computer scientist at the University of Durham in the UK. ‘If we can keep activities like egg laying happening at the right time, we are expecting to have healthier broods and more active and healthy colonies. This will then improve pollination.’

While that goes on above the surface, shape-morphing robot roots that can adapt and interact with real plants and fungi are hard at work underground. There, plants and their fungal partners form vast networks.

These robotic bees and roots are being developed by two EU-funded projects. Both initiatives are looking into how artificial versions of living things central to maintaining ecosystems can help real-life organisms and their environment survive and thrive – while ensuring food for people remains plentiful.

“If we can keep activities like egg laying happening at the right time, we are expecting to have healthier broods.”

– Dr Farshad Arvin, RoboRoyale

That could be crucial to the planet’s long-term future, particularly with many species currently facing steep population declines as a result of threats that include habitat loss, pollution and climate change.

One of those at risk is the honeybee, a keystone species in the insect pollination required for 75% of crops grown for human food globally.

Fit for a queen

The RoboRoyale project that Arvin leads combines microrobotic, biological and machine-learning technologies to nurture the queen honeybee’s well-being. The project is funded by the European Innovation Council’s Pathfinder programme.

A unique aspect of RoboRoyale is its sole focus on the queen rather than the entire colony, according to Arvin. He said the idea is to demonstrate how supporting a single key organism can stimulate production in the whole environment, potentially affecting hundreds of millions of organisms.

The multi-robot system, which the team hopes to start testing in the coming months, will learn over time how to groom the queen to optimise her egg laying and production of pheromones – chemical scents that influence the behaviour of the hive.

The system is being deployed in artificial glass observation hives in Austria and Turkey, with the bee replicants designed to replace the so-called court bees that normally interact with the queen.

Foods for broods

One aim is that the robot bees can potentially stimulate egg laying by providing the queen with specific protein-rich foods at just the right time to boost this activity. In turn, an expected benefit is that a resulting increase in bees and foraging flights would mean stronger pollination of the surrounding ecosystem to support plant growth and animals.

The system enables six to eight robotic court bees, some equipped with microcameras, to be steered inside an observation hive by a controller attached to them from outside. The end goal is to make the robot bees fully autonomous.

The concept design of RoboRoyale robotic controller. © Farshad Arvin, 2023

Prior to this, the RoboRoyale team observed queen bees in several hives using high-resolution cameras and image-analysis software to get more insight into their behaviour.

The team captured more than 150 million samples of the queens’ trajectories inside the hive and detailed footage of their social interactions with other bees. It is now analysing the data.

Once the full robotic system is sufficiently tested, the RoboRoyale researchers hope it will foster understanding of the potential for bio-hybrid technology not only in bees but also in other organisms.

‘It might lead to a novel type of sustainable technology that positively impacts surrounding ecosystems,’ said Arvin.

Wood Wide Web

The other project, I-Wood, is exploring a very different type of social network – one that’s underground.

Scientists at the Italian Institute of Technology (IIT) in Genoa are studying what they call the Wood Wide Web. It consists of plant roots connected to each other through a symbiotic network of fungi that provide them with nutrients and help them to share resources and communicate.

“Biomimicry in robotics and technology will have a fundamental role in saving our planet.”

– Dr Barbara Mazzolai, I-Wood

To understand these networks better and find ways to stimulate their growth, I-Wood is developing soft, shape-changing robotic roots that can adapt and interact with real plants and fungi. The idea is for a robotic plant root to use a miniaturised 3D printer in its tip to enable it to grow and branch out, layer by layer, in response to environmental factors such as temperature, humidity and available nutrients.

‘These technologies will help to increase knowledge about the relationship between symbionts and hosts,’ said Dr Barbara Mazzolai, an IIT roboticist who leads the project.

Mazzolai’s team has a greenhouse where it grows rice plants inoculated with fungi. So far, the researchers have separately examined the growth of roots and fungi.

Soon, they plan to merge their findings to see how, when and where the interaction between the two occurs and what molecules it involves.

The findings can later be used by I-Wood’s robots to help the natural symbiosis between fungi and roots work as effectively as possible. The team hopes to start experimenting with robots in the greenhouse by the end of this year.

The robotic roots can be programmed to move autonomously, helped by sensors in their tips, according to Mazzolai. Like the way real roots or earthworms move underground, they will also seek passages that are easier to move through due to softer or less compact soil.

Tweaks of the trade

But there are challenges in combining robotics with nature.

For example, bees are sensitive to alien objects in their hive and may remove them or coat them in wax. This makes it tricky to use items like tracking tags.

The bees have, however, become more accepting after the team tweaked elements of the tags such as their coating, materials and smell, according to Arvin of RoboRoyale.

Despite these challenges, Arvin and Mazzolai believe robotics and artificial intelligence could play a key part in sustaining ecosystems and the environment in the long term. For Mazzolai, the appeal lies in the technologies’ potential to offer deeper analysis of little-understood interactions among plants, animals and the environment.

For instance, with the underground web of plant roots and fungi believed to be crucial to maintaining healthy ecosystems and limiting global warming by locking up carbon, the project’s robotic roots can help shed light on how we can protect and support these natural processes.

‘Biomimicry in robotics and technology will have a fundamental role in saving our planet,’ Mazzolai said.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Researchers are ushering in a new way of thinking about robots in the workplace based on the idea of robots and workers as teammates rather than competitors. © BigBlueStudio, Shutterstock

For decades, the arrival of robots in the workplace has been a source of public anxiety over fears that they will replace workers and create unemployment.

Now that more sophisticated and humanoid robots are actually emerging, the picture is changing, with some seeing robots as promising teammates rather than unwelcome competitors.

‘Cobot’ colleagues

Take Italian industrial-automation company Comau. It has developed a robot that can collaborate with – and enhance the safety of – workers in strict cleanroom settings in the pharmaceutical, cosmetics, electronics, food and beverage industries. The innovation is known as a “collaborative robot”, or “cobot”.

Comau’s arm-like cobot, which is designed for handling and assembly tasks, can automatically switch from an industrial to a slower speed when a person enters the work area. This new feature allows one robot to be used instead of two, maximising productivity and protecting workers.

‘It has advanced things by allowing a dual mode of operation,’ said Dr Sotiris Makris, a roboticist at the University of Patras in Greece. ‘You can either use it as a conventional robot or, when it is in collaborative mode, the worker can grab it and move it around as an assisting device.’

Makris was coordinator of the just-completed EU-funded SHERLOCK project, which explored new methods for safely combining human and robot capabilities from what it regarded as an often overlooked research angle: psychological and social well-being.

Creative and inclusive

Robotics can help society by carrying out repetitive, tedious tasks, freeing up workers to engage in more creative activities. And robotic technologies that can collaborate effectively with workers could make workplaces more inclusive, such as by aiding people with disabilities.

“There is increasing competition around the globe, with new advances in robotics.”

– Dr Sotiris Makris, SHERLOCK

These opportunities are important to seize as the structure and the age profile of the European workforce changes. For example, the proportion of 55-to-64-year-olds increased from 12.5% of the EU’s employees in 2009 to 19% in 2021.

Alongside the social dimension, there is also economic benefit from greater industrial efficiency, showing that neither necessarily needs to come at the expense of the other.

‘There is increasing competition around the globe, with new advances in robotics,’ said Makris. ‘That is calling for actions and continuous improvement in Europe.’

Makris cites the humanoid robots being developed by Elon Musk-led car manufacturer Tesla. Wearable robotics, bionic limbs and exoskeleton suits are also being developed that promise to enhance people’s capabilities in the workplace.

Still, the rapidly advancing wave of robotics poses big challenges when it comes to ensuring they are effectively integrated into the workplace and that people’s individual needs are met when working with them. 

Case for SHERLOCK

SHERLOCK also examined the potential for smart exoskeletons to support workers in carrying and handling heavy parts at places such as workshops, warehouses or assembly sites. Wearable sensors and AI were used to monitor and track human movements.

With this feedback, the idea is that the exoskeleton can then adapt to the needs of the specific task while helping workers retain an ergonomic posture to avoid injury.

‘Using sensors to collect data from how the exoskeleton performs allowed us to see and better understand the human condition,’ said Dr Makris. ‘This allowed us to have prototypes on how exoskeletons need to be further redesigned and developed in the future, depending on different user profiles and different countries.’

SHERLOCK, which has just ended after four years, brought together 18 European organisations in multiple countries from Greece to Italy and the UK working on different areas of robotics.

The range of participants enabled the project to harness a wide variety of perspectives, which Dr Makris said was also beneficial in the light of differing national rules on integrating robotics technology.

As a result of the interaction of these robotic systems with people, the software is advanced enough to give direction to ‘future developments on the types of features to have and how the workplace should be designed,’ said Dr Makris.

Old hands, new tools

Another EU-funded project that ended this year, CO-ADAPT, used cobots to help older people navigate the digitalised workplace.

“You find interesting differences in how much the machine and how much the person should do.”

– Prof Giulio Jacucci, CO-ADAPT

The project team developed a cobot-equipped adaptive workstation to aid people in assembly tasks, such as making a phone, car or toy – or, indeed, combining any set of individual components into a finished product during manufacturing. The station can adapt workbench height and lighting to a person’s physical characteristics and visual abilities. It also includes features like eye-tracking glasses to gather information on mental workload.

That brings more insight into what all kinds of people need, said Professor Giulio Jacucci, coordinator of CO-ADAPT and a computer scientist at the University of Helsinki in Finland.

‘You find interesting differences in how much the machine and how much the person should do, as well as how much the machine should try to give guidance and how,’ Jacucci said. ‘This is important work that goes down to the nuts and bolts of making this work.’

Still, cobot-equipped workplaces that can fully tap into and respond to people’s mental states in real-life settings could still be a number of years away, he said.

‘It’s so complex because there’s the whole mechanical part, plus trying to understand people’s status from their psychophysiological states,’ said Prof Jacucci.

Meanwhile, because new technologies can be used in much simpler ways to improve the workplace, CO-ADAPT also explored digitalisation more broadly.

Smart shifts

One area was software that enables ‘smart-shift scheduling’, which arranges duty periods for workers based on their personal circumstances. The approach has been shown to reduce sick leave, stress and sleep disorders among social welfare and health care workers.

‘It’s a fantastic example of how workability improves because we use evidence-based knowledge of how to have well-being-informed schedules,’ said Prof Jacucci.

Focusing on the individual is key to the future of well-integrated digital tools and robotics, he said.

‘Let’s say you have to collaborate with some robot in an assembly task,’ he said. ‘The question is: should the robot be aware of my cognitive and other abilities? And how should we divide the task between the two?’

The basic message from the project is that plenty of room exists to improve and broaden working environments.

‘It shows how much untapped potential there is,’ said Prof Jacucci.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

Artificial intelligence is set to revolutionise agriculture by helping farmers meet field-hand needs and identify diseased plants. © baranozdemir, iStock

In the Dutch province of Zeeland, a robot moves swiftly through a field of crops including sunflowers, shallots and onions. The machine weeds autonomously – and tirelessly – day in, day out.

“Farmdroid” has made life a lot easier for Mark Buijze, who runs a biological farm with 50 cows and 15 hectares of land. Buijze is one of the very few owners of robots in European agriculture.

Robots to the rescue

His electronic field worker uses GPS and is multifunctional, switching between weeding and seeding. With the push of a button, all Buijze has to do is enter coordinates and Farmdroid takes it from there.

‘With the robot, the weeding can be finished within one to two days – a task that would normally take weeks and roughly four to five workers if done by hand,’ he said. ‘By using GPS, the machine can identify the exact location of where it has to go in the field.’

About 12 000 years ago, the end of foraging and start of agriculture heralded big improvements in people’s quality of life. Few sectors have a history as rich as that of farming, which has evolved over the centuries in step with technological advancements.

In the current era, however, agriculture has been slower than other industries to follow one tech trend: artificial intelligence (AI). While already commonly used in forms ranging from automated chatbots and face recognition to car braking and warehouse controls, AI for agriculture is still in the early stages of development.

Now, advances in research are spurring farmers to embrace robots by showing how they can do everything from meeting field-hand needs to detecting crop diseases early.

Lean and green

For French agronomist Bertrand Pinel, farming in Europe will require far greater use of robots to be productive, competitive and green – three top EU goals for a sector whose output is worth around €190 billion a year.

“Labour is one of the biggest obstacles in agriculture.”

– Fritz van Evert, ROBS4CROPS

One reason for using robots is the need to forgo the use of herbicides by eliminating weeds the old-fashioned way: mechanical weeding, a task that is not just mundane but also arduous and time consuming. Another is the frequent shortage of workers to prune grapevines.

‘In both cases, robots would help,’ said Pinel, who is research and development project manager at France-based Terrena Innovation. ‘That is our idea of the future for European agriculture.’

Pinel is part of the EU-funded ROBS4CROPS project. With some 50 experts and 16 institutional partners involved, it is pioneering a robot technology on participating farms in the Netherlands, Greece, Spain and France.

‘This initiative is quite innovative,’ said Frits van Evert, coordinator of the project. ‘It has not been done before.’

In the weeds

AI in agriculture looks promising for tasks that need to be repeated throughout the year such as weeding, according to van Evert, a senior researcher in precision agriculture at Wageningen University in the Netherlands.

‘If you grow a crop like potatoes, typically you plant the crop once per year in the spring and you harvest in the fall, but the weeding has to be done somewhere between six and 10 times per year,’ he said.

Plus, there is the question of speed. Often machines work faster than any human being can.

“With this robot everything is done in the field.”

– Francisco Javier Nieto De Santos, FLEXIGROBOTS

Francisco Javier Nieto De Santos, coordinator of the EU-funded FLEXIGROBOTS project, is particularly impressed by a model robot that takes soil samples. When done by hand, this practice requires special care to avoid contamination, delivery to a laboratory and days of analysis.

‘With this robot everything is done in the field,’ De Santos said. ‘It can take several samples per hour, providing results within a matter of minutes.’

Eventually, he said, the benefits of such technologies will extend beyond the farm industry to reach the general public by increasing the overall supply of food.

Unloved labour

Meanwhile, agricultural robots may be in demand not because they can work faster than any person but simply because no people are available for the job.

Even before inflation rates and fertiliser prices began to surge in 2021 amid an energy squeeze made worse by Russia’s invasion of Ukraine this year, farmers across Europe were struggling on another front: finding enough field hands including seasonal workers.

‘Labour is one of the biggest obstacles in agriculture,’ said van Evert. ‘It’s costly and hard to get these days because fewer and fewer people are willing to work in agriculture. We think that robots, such as self-driving tractors, can take away this obstacle.’

The idea behind ROBS4CROPS is to create a robotic system where existing agricultural machinery is upgraded so it can work in tandem with farm robots.

For the system to work, raw data such as images or videos must first be labelled by researchers in ways than can later be read by the AI.

Driverless tractors

The system then uses these large amounts of information to make “smart” decisions as well as predictions – think about the autocorrect feature on laptop computers and mobile phones, for example.

A farming controller comparable to the “brain” of the whole operation decides what needs to happen next or how much work remains to be done and where – based on information from maps or instructions provided by the farmer.

The machinery – self-driving tractors and smart implements like weeders equipped with sensors and cameras – gathers and stores more information as it works, becoming “smarter”.

Crop protection

FLEXIGROBOTS, based in Spain, aims to help farmers use existing robots for multiple tasks including disease detection.

Take drones, for example. Because they can spot a diseased plant from the air, drones can help farmers detect sick crops early and prevent a wider infestation.

‘If you can’t detect diseases in an early stage, you may lose the produce of an entire field, the production of an entire year,’ said De Santos. ‘The only option is to remove the infected plant.’

For example, there is no treatment for the fungus known as mildew, so identifying and removing diseased plants early on is crucial.

Pooling information is key to making the whole system smarter, De Santos said. Sharing data gathered by drones with robots or feeding the information into models expands the “intelligence” of the machines.

Although agronomist Pinel doesn’t believe that agriculture will ever be solely reliant on robotics, he’s certain about their revolutionary impact.

‘In the future, we hope that the farmers can just put a couple of small robots in the field and let them work all day,’ he said.

Research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

Watch the video

This article was originally published in Horizon, the EU Research and Innovation magazine.

The Internet of Things allows our smart gadgets in the home and wearable technologies like our smart watches to communicate and operate together. Image Credit: Ponchai nakumpa via Pixabay

Tiny internet-connected electronic devices are becoming ubiquitous. The so-called Internet of Things (IoT) allows our smart gadgets in the home and wearable technologies like our smart watches to communicate and operate together. IoT devices are increasingly used across all sorts of industries to drive interconnectivity and smart automation as part of the ‘fourth industrial revolution’.

The fourth industrial revolution builds on already widespread digital technology such as connected devices, artificial intelligence, robotics and 3D printing. It is expected to be a significant factor in revolutionising society, the economy and culture.

These small, autonomous, interconnected and often wireless devices are already playing a key role in our everyday lives by helping to make us more resource and energy-efficient, organised, safe, secure and healthy.

There is a key challenge, however – how to power these tiny devices. The obvious answer is “batteries”. But it is not quite that simple.

Small devices

Many of these devices are too small to use a long-life battery and they are located in remote or hard-to-access locations – for instance in the middle of the ocean tracking a shipping container or at the top of a grain silo, monitoring levels of cereal. These types of locations make servicing some IoT devices extremely challenging and commercially and logistically infeasible.

Mike Hayes, head of ICT for energy efficiency at the Tyndall National Institute in Ireland, summarises the marketplace. ‘It’s projected that we are going to have one trillion sensors in the world by 2025,’ he said, ‘That is one thousand billion sensors.’

That number is not as crazy as it first seems, according to Hayes, who is the coordinator of the Horizon-funded EnABLES project (European Infrastructure Powering the Internet of Things).

If you think about the sensors in the technology someone might carry on their person or have in their car, home, office plus the sensors embedded in the infrastructure around them such as roads and railways, you can see where that number comes from, he explained.

“In the trillion IoT sensor world predicted for 2025, we are going to be throwing over 100 million batteries everyday into landfills unless we significantly extend battery life.”

– Mike Hayes, EnABLES

Battery life

Landfill is not the only environmental concern. We also need to consider where all the material to make the batteries is going to come from. The EnABLES project is calling on the EU and industry leaders to think about battery life from the outset when designing IoT devices to ensure that batteries are not limiting the lifespan of devices.

‘We don’t need the device to last forever,’ said Hayes. ‘The trick is that you need to outlive the application that you’re serving. For example, if you want to monitor a piece of industrial equipment, you probably want it to last for five to 10 years. And in some cases, if you do a regular service every three years anyway, once the battery lasts more than three or four years that’s probably good enough.’

Although many devices have an operational life of more than 10 years, the battery life of wireless sensors is typically only one to two years.

The first step to longer battery life is increasing the energy supplied by batteries. Also, reducing the power consumption of devices will prolong the battery. But EnABLES is going even further.

The project brings together 11 leading European research institutes. With other stakeholders, EnABLES is working to develop innovative ways to harvest tiny ambient energies such as light, heat and vibration.

Harvesting such energies will further extend battery life. The goal is to create self-charging batteries that last longer or ultimately run autonomously.

Energy harvesters

mbient energy harvesters, such as a small vibrational harvester or indoor solar panel, that produce low amounts of power (in the milliwatt range) could significantly extend the battery life of many devices, according to Hayes. These include everyday items like watches, radio frequency identification (RFID) tags, hearing aids, carbon dioxide detectors, and temperature, light and humidity sensors.

EnABLES is also designing the other key technologies needed for tiny IoT devices. Not content with improving energy efficiency, the project is also trying to develop a framework and standardised and interoperable technologies for these devices.

One of the key challenges with autonomously powered IoT tools is power management. The energy source may be intermittent and at very low levels (microwatts), and different methods of harvesting supply different forms of power that require different techniques to convert to electricity.

Steady trickle

Huw Davies, is chief executive officer of Trameto, a company which is developing power management for piezo electric applications. He points out that energy from photovoltaic devices tends to come in a steady trickle, while that from piezoelectric devices, which convert ambient energy from movements (vibrations) into electrical energy, generally comes in bursts.

‘You need a way of storing that energy locally in a store before it is delivered into a load, so you need to have ways of managing that,’ Davies said.

He is the project coordinator of the Horizon-funded HarvestAll project, which has developed an energy management system for ambient energy dubbed OptiJoule.

OptiJoule works with piezoelectric materials, photovoltaics and thermal electric generators. It can function with any of these sources on their own, or with multiple energy harvesting sources at the same time.

The goal is to enable autonomous sensors to be self-sustaining. In principle, it’s quite simple. ‘What we are talking about is ultra-low powered sensors taking some digital measurement,’ said Davies. ‘Temperature, humidity, pressure, whatever it is, with the data from that being delivered into the internet.’

Integrated circuits

The HarvestAll energy management integrated circuit device adjusts to match the different energy harvesters. It takes the different and intermittent energy created by these harvesters and stores it, for instance in a battery or capacitor, and then manages the delivery of a steady output of energy to the sensor.

Similarly to the EnABLES project, the idea is to create standardised technology that will enable the rapid development of long battery life/autonomous IoT devices in Europe and the world.

Davies said that the energy management circuit works completely autonomously and automatically. It is designed so that it can just be plugged into an energy harvester, or combination of harvesters, and a sensor. As a replacement for the battery it has a significant advantage, according to Davies, because ‘It will just work.’

Research in this article was funded by the EU.

This article was originally published in Horizon, the EU Research and Innovation magazine.

A sustainable and scalable gas fermentation technology transforms CO2 from industrial emissions into a single cell protein for animal nutrition. © Valdis Skudre, Shutterstock

By Andrew Dunne

Gardening is proven to be healthful and joyful, but as more of us discover the joys of working in the garden for the first time, some basic knowledge about plants, landscaping and soil is required to get started. What, where and when should you plant, for instance?

These were some of the core questions co-founder of the start-up Draw Me A Garden (DMAG), Florent De Salaberry, realised were standing in the way of more people digging in to the subject.

IKEA-like

Many people want to garden, but lots of us just don’t have the expertise or confidence to begin,’ said the French tech entrepreneur.

DMAG is an app and website service which offers tailored 3D-plans for garden design. It helps budding gardeners to transform any plot into a beautiful, sustainable garden with ease.

The inspiration behind the company’s name comes from the children’s book ‘Le Petit Prince’ in which the prince requests the narrator to ‘draw me a sheep’ to start a conversation and build a relationship.

De Salaberry says “Draw Me A Garden” uses digital tools in a similar way to help people build a relationship with nature in their gardens.

The DMAG service helps customers envisage their dream garden by providing creative ideas, planting tips and, most important of all, delivering all the plants to their door.

Giving customers ownership of their creations is what distinguishes DMAG from traditional landscaping, argues De Salaberry. ‘We know that if you just pay people to landscape your garden, not only is that really expensive but it’s also hard to feel pride in it,’ he said.

‘DMAG is about making gardening easy and affordable, and providing the resources to enable customers to be at the heart of their own projects.’

Garden varieties

Customers locate their garden online via a satellite map. Next, they list any pre-existing features such as a terrace or a child’s play area, then select a preferred garden style, such as for example English cottage garden or Mediterranean.

“Many people want to garden, but lots of us just don’t have the expertise or confidence to begin.”

– Florent de Salaberry, Draw Me A Garden

Behind the scenes, DMAG’s algorithm whirrs away using these inputs together with local knowledge (soil type, elevation, sun direction) to map out the perfect garden design. Customers can visualise the design using 3D mapping tools on the DMAG website.

A qualified landscaper supports the design process and the customer receives a number of planning options to mull over.

Green thumbs

Results come back almost instantaneously. ‘The idea was always to enable customers to do this wherever or whenever they wanted and it takes just a few seconds to get the first design back,’ said De Salaberry.

Once further small refinements are made, a 3D view is rendered, and customers can sit back and wait for all plants and growing instructions to be delivered.

A typical delivery might consist of between 200 – 300 plants. These come with biodegradable cardboard scaffolds cut to the exact garden size and instructions to help the gardeners plant them out.

So far, the DMAG team have supplied to gardeners of all kinds in France, Belgium and Luxembourg, with average expenditure of around €1 500.

De Salaberry likens his turnkey garden concept to how IKEA has revolutionised kitchen design.

As they look to scale-up this work in new EU countries and the US, they hope many more people will soon be asking them to start their gardening journey and “draw me a garden.”

Glade runner

If DMAG can help gardeners create the ideal future garden space, then the TrimBot2020 might be the answer to help maintain it.

The brainchild of computer vision and robotics’ expert, Professor Bob Fisher of the University of Edinburgh, TrimBot2020 is one of the first robot gardening devices that promises to do more than simply mow the lawn.

The TrimBot2020 © TrimBot2020 Consortium, 2020

Based on a modified commercially available robot lawnmower, the autonomous vehicle prunes roses, trims hedges and shapes topiary, all while auto-navigating garden terrain.

To achieve this, the robot uses a ring of cameras to draw a 3D map of the garden, some robotic snippers and hefty dose of computer processing power.

‘There are ten cameras which work together to build up a 3D model of the garden, just like our eyes do,’ said Fisher.

Together, these cameras help the robot gain a 360-degree view of the complex terrain of the garden. The robot also matches what it sees to a hand drawn map supplied by the users.

Upon command, the TrimBot springs into life by rolling up to the bush and scanning it to build up a computer-vision model of that particular plant.

‘Once it has an idea of where all the stems are, its robotic arm comes out with the cutter and it starts snipping away,’ said Fisher.

Robochop

For the TrimBot team, the commercial target market is horticultural businesses responsible for maintaining parks, gardens, and recreational areas.

In such cases, they believe the robot can take on pruning duties while the human gardener does something more challenging.

While the commercial future of TrimBot is yet to be determined, the real benefits may yet come through incorporating the technology into the “brains” of next-generation of garden robots.

‘Outdoor robotics is notoriously hard,’ said Fisher. Typical challenges include constant lighting changes, the many different shades of green and variations in the terrain.

Current robot lawnmowers usually require users to mark out an exact area to mow and to position a robot in the right place to start. TrimBot’s technology should enable robots of tomorrow to work that out themselves.

‘With the TrimBot project we’ve really demonstrated what might be possible in the future,’ said Fisher.

Research in this article was funded by the EU.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Uncrewed aircraft in the sky above the headquarters of the Port of Antwerp-Bruges. © Helicus – Geert Vanhandenhove, Rik Van Boxem, 2022

By Gareth Willmer

In a city in the future, a fire breaks out in a skyscraper. An alarm is triggered and a swarm of drones swoops in, surrounds the building and uses antennas to locate people inside, enabling firefighters to go straight to the stricken individuals. Just in the nick of time – no deaths are recorded.

Elsewhere in the city, drones fly back and forth delivering tissue samples from hospitals to specialist labs for analysis, while another rushes a defibrillator to someone who has suffered a suspected cardiac arrest on a football pitch. The patient lives, with the saved minutes proving critical.

At the time of writing, drones have already been used in search-and-rescue situations to save more than 880 people worldwide, according to drone company DJI. Drones are also being used for medical purposes, such as to transport medicines and samples, and take vaccines to remote areas.

Drones for such uses are still a relatively new development, meaning there is plenty of room to make them more effective and improve supporting infrastructure. This is particularly true when it comes to urban environments, where navigation is complex and requires safety regulations.

Flying firefighters

The IDEAL DRONE project developed a system to aid in firefighting and other emergencies to demonstrate the potential for using swarms of uncrewed aerial vehicles (UAVs) in such situations. Equipped with antennas, the drones use a radio-frequency system to detect the location of ‘nodes’ – or tags – worn by people inside a building.

“By knowing how many people are inside the building and where they are located, it will optimise the search-and-rescue operation.”

– Prof Gian Paolo Cimellaro, IDEAL DRONE

Making use of an Italian aircraft hangar, the tests involved pilots on the ground flying three drones around the outside of a building. The idea is that the drones triangulate the position of people inside where their signals intersect, as well as detecting information about their health condition. The details can then be mapped to optimise and accelerate rescue operations, and enhance safety for firefighters by allowing them to avoid searching all over a burning building without knowing where people are.

‘You create a sort of temporary network from outside the building through which you can detect the people inside,’ said Professor Gian Paolo Cimellaro, an engineer at the Polytechnic University of Turin and project lead on IDEAL DRONE.

‘By knowing how many people are inside the building and where they are located, it will optimise the search-and-rescue operation.’

He added: ‘A unique characteristic of this project is that it allows indoor tracking without communication networks such as Wi-Fi or GPS, which might not be available if you are in an emergency like a disaster or post-earthquake situation.’

There are some challenges in terms of accuracy and battery life, while another obvious drawback is that people in the building need to already be wearing trackers.

However, said Prof Cimellaro, current thinking is that this can be unintrusive if tags are incorporated in existing technology that people often already carry such as smartwatches, mobile phones or ID cards. They can also be used by organisations that mandate their use for staff working in hazardous environments, such as factories or offshore oil rigs.

Looking beyond the challenges, Prof Cimellaro thinks such systems could be a reality within five years, with drones holding significant future promise for avoiding ‘putting human lives in danger’.

Medical networks

Another area in which drones can be used to save lives is medical emergencies. This is the focus of the SAFIR-Med project.

Belgian medical drone operator Helicus has established a command-and-control (C2C) centre in Antwerp to coordinate drone flights. The idea is that the C2C automatically creates flight plans using artificial intelligence, navigating within a digital twin – or virtual representation – of the real world. These plans are then relayed to the relevant air traffic authorities for flight authorisation.

‘We foresee drone cargo ports on the rooftops of hospitals, integrated as much as possible with the hospital’s logistical system so that transport can be on demand,’ added Geert Vanhandenhove, manager of flight operations at Helicus.

So far, SAFIR-Med has successfully carried out remote virtual demonstrations, simulations, flights controlled from the C2C at test sites, and other tests such as that of a ‘detect-and-avoid’ system to help drones take evasive action when others are flying in the vicinity.

The next step will be to validate the concepts in real-life demonstrations in several countries, including Belgium, Germany and the Netherlands. The trials envisage scenarios including transfers of medical equipment and tissue samples between hospitals and labs, delivery of a defibrillator to treat a cardiac patient outside a hospital, and transport of a physician to an emergency site by passenger drone.

“We foresee drone cargo ports on the rooftops of hospitals.”

– Geert Vanhandenhove, SAFIR-Med

Additional simulations in Greece and the Czech Republic will show the potential for extending such systems across Europe.

SAFIR-Med is part of a wider initiative known as U-space. It’s co-funded by the Single European Sky Air Traffic Management Research (SESAR) Joint Undertaking which is a public-private effort for safer drone operations under the Digital European Sky.

Making rules

Much of the technology is already there for such uses of drones, says Vanhandenhove. However, he highlights that there are regulatory challenges involved in drone flights in cities, especially with larger models flying beyond visual line of sight (BVLOS). This includes authorisations for demonstrations within SAFIR-Med itself.

‘The fact that this is the first time this is being done is posing significant hurdles,’ he said. ‘It will depend on the authorisations granted as to which scenarios can be executed.’

But regulations are set to open up over time, with European Commission rules facilitating a framework for use of BVLOS UAVs in low-level airspace due to come into force next January.

Vanhandenhove emphasises that the development of more robust drone infrastructure will be a gradual process of learning and improvement. Eventually, he hopes that through well-coordinated systems with authorities, emergency flights can be mobilised in seconds in smart cities of the future. ‘For us, it’s very important that we can get an authorisation in sub-minute time,’ he said.

He believes commercial flights could even begin within a couple of years, though it may not be until post-2025 that widely integrated, robust uncrewed medical systems come into play in cities. ‘It’s about making the logistics of delivering whatever medical treatment faster and more efficient, and taking out as much as possible the constraints and limitations that we have on the route,’ said Vanhandenhove.

Research in this article was funded via the EU’s European Research Council.

This article was originally published in Horizon, the EU Research and Innovation magazine.

Innovative technologies are under development to reduce plastic litter at sea by at least 50% © MOHAMED ABDULRAHEEM, Shutterstock

By Gareth Willmer

‘It’s the scale of it – it’s a global problem. You can guarantee that any beach you walk on, you’ll find pieces of plastic,’ said James Comerford, a senior researcher in materials and nanotechnology at SINTEF, an independent research organisation in Oslo, Norway.

Plastics are estimated to comprise 85% of marine litter, with 11 million metric tonnes entering the oceans annually and the volume potentially tripling by 2040. Some have predicted that, by weight, there will be more plastics than fish in the seas by 2050.

In light of the alarming outlook, innovative approaches are required to tackle the problem. This is exactly what the EU Mission “Restore our Ocean and Waters by 2030” is targeting, with the ambition of reducing plastic litter at sea by at least 50%, cutting microplastics released into the environment by 30%, and halving agricultural nutrient losses as well as the use of chemical pesticides.

To reduce pollution, the Mission is launching a ‘lighthouse’ in the Mediterranean Sea that will act as a hub to develop, demonstrate and deploy solutions far and wide across the world by getting all the relevant players on board. Its role is to connect and structure activities, disseminate and upscale solutions and mobilise relevant actors.

Its initial focus is on plastic pollution. Projects such as In-No-Plastic and AQUA-LIT are exploring ways to reduce the contribution of people and sea-based industries to plastic pollution, while the Maelstrom project looks at where marine debris is distributed and how best to remove it from the seabed and water. It is also exploring economically viable ways to recover and recycle marine plastic debris, such as circular product design for fishing gear.

The wide-reaching In-No-Plastic project, led by Comerford as the project coordinator, is developing a range of technologies that deal not only with easily visible, large pieces of plastics – or macroplastics – but also the insidious threat of tiny microplastics measuring less than 5 millimetres, and even smaller nanoplastics.

‘Macroplastics are going to need different cleaning technologies to microplastics, so we’re looking at the whole spectrum,’ said Comerford.

Several separate technologies that are currently under development can be deployed in tandem to clean up the water. A couple of them help to deal with microplastics by clumping them into more manageable sizes, one using biodegradable chemical substances called flocculants that cause particles to coagulate, and the other – known as SepaRaptor – using ultrasonic waves that push the particles into clusters.

These can be combined with another technology that uses a screen to sift out plastic debris.

On the macroplastics side of things is SEEker, a four-wheeled plastic-waste-collection robot being trained using artificial intelligence to identify and pick up litter from beaches and put it in a bin carried on its back. The robot will also have a loading station near the beach, where it can dispose of waste and recharge.

‘It’ll be entirely autonomous,’ said Comerford. ‘Because there’s so much litter and because it’s everywhere, you need something focusing on it all the time. To have solely a human influence is really time-consuming.’

Mobile application

Another technology, which includes features that could be key to tackling the issue of plastic pollution in the long-term future, is an application for smartphones. This encourages volunteers to gather litter and record data on their activities, using “social rewards” sourced via the local economy – for example, discounts on pizzas or at the gym.

However, the app will also eventually help to track the amount of plastic waste collected, recycled and used in products, allowing us to get more of a handle on how effectively the circular economy is working.

So many people say they include recycled material in products. If we’re really to make a difference and turn this whole thing round, that’s got to be countable

Although that function is currently under development, Comerford explained that it will be supported using photos and GPS data on collected litter, as well as blockchain technology – which can enable better tracing of the contents of goods by storing data on the movement of materials through a supply chain.

‘So many people say they include recycled material in products,’ said Comerford. ‘If we’re really to make a difference and turn this whole thing round, that’s got to be countable.’

But apart from the pure tech side, public buy-in for solutions to the plastic problem is crucial. Partners in In-No-Plastic, such as non-profit organisation Venice Lagoon Plastic Free (VLPF), are also conducting clean-up initiatives supported by the mobile application and gauging the attitudes of the public on plastic pollution.

Davide Poletto, an executive director at the organisation, says Venice is an ideal place to run plastic pollution initiatives, as a location with an enclosed area of water, and intense marine traffic, aquaculture, fishery activity and tourism. ‘The lagoon of Venice is the largest wetland in the Mediterranean basin and a World Heritage Site of UNESCO, and this is an extraordinary laboratory to work in because you have a lot of different problems,’ he said.

He also points out that the pandemic has provided a ‘unique opportunity’ to analyse just how much overtourism contributes to pollution, including that caused by plastics, and the capacity of the local ecosystem to recover. Poletto cites a study showing that 17 of 40 chemical contaminants previously found in the Venice lagoon were undetectable after early-2020 lockdowns, while the presence of many others was significantly lower.

Boosting awareness

Recent In-No-Plastic events appear to have shown promise for growing public awareness and interest in getting involved. In one clean-up event organised in Venice in 2021, 130 people collected three tonnes of waste, including more than 1,500 kilograms of plastics.

Poletto also cites figures from an ongoing awareness study carried out by his team on more than 1,500 people in Italy, the UK and Croatia, the vast majority from outside related work sectors. Over 85% of respondents per country said joining clean-up events had helped them better understand the seriousness of marine plastic pollution, while almost 95% identified microplastics as a bigger issue than macroplastics – suggesting understanding is now widespread on the perils of invisible fragments.

Poletto pointed to growing coverage in the news and social media, as well as first-hand experience. ‘It’s interesting to see how people are realising all those things,’ he said. ‘And it’s not that they are specialists.’

But apart from stimulating public interest, he said more knowledge is needed on sources of plastic pollution to better advise decision-makers on how to deal with it. Using another app that aids with beach litter identification as part of the Maelstrom project, VLPF found that on some beaches, up to 40% of plastics on nearby islands such as Pellestrina came from fishing gear – mostly mussel nets.

This is important to show, for instance, that a big proportion of plastics in these areas goes straight into the sea rather than originating in rivers, said Poletto. ‘Then there’s evidence brought to the public administration that we should do more in certain locations.’

Aquaculture challenge

Gear is a big issue in the aquaculture industry too, where there is also an urgent need to tackle plastic pollution given that it is the world’s fastest-growing food sector. Aquaculture is estimated to account for more than half of global fish consumption, and could reach over 60% in the next decade.

But Mariana Mata Lara, project manager at environmental technology organisation Geonardo, says that much more knowledge is needed on how to tackle plastic pollution from the sector, caused by items including cages, ropes, nets and buoys.

She also said we need to separate data on pollution caused by aquaculture, or farming of aquatic produce, from that caused by traditional fisheries that catch wild fish. ‘In reality, we don’t know exactly the amount of plastics that comes from this sector,’ added Lara.

With this in mind, a project she led called AQUA-LIT sought to create a knowledge base on both plastics and other marine waste before the problem gets too big as the sector surges. ‘In many things in life, we come up with solutions once the problem exists. The idea with AQUA-LIT was to go in parallel and start solving this as it grows, so we don’t later have to come up with solutions to cover what we did in the past,’ said Lara.

AQUA-LIT did this by developing a toolbox of measures to monitor and prevent marine littering in the sector, as well as to remove and recycle waste.

The idea with AQUA-LIT was to go in parallel and start solving this as it grows, so we don’t later have to come up with solutions to cover what we did in the past

The team gathered the information by working with research institutes, organisations and people involved in aquaculture in the Mediterranean, North Sea and Baltic Sea. Activities included interactive ‘Learning Lab’ workshops to discuss marine litter issues, exchange knowledge and brainstorm ideas.

More than 400 ideas and solutions

The resulting toolbox contains a variety of measures, arranged by topics including different sea basins, aquaculture types, and stage of removal and recycling, as well as policy recommendations. ‘In the toolbox, we have provided more than 400 ideas and solutions,’ said Lara.

As part of its work, AQUA-LIT has created an inventory detailing 65 sources of waste generated by aquaculture, a database on how European ports deal with litter and regional maps on percentages of aquaculture-related litter across its focus sea basins.

Lara added that many of these ideas can be applied or expanded on elsewhere. ‘We wanted this information to be useful not only for these three sea basins we worked in, so we created action plans to transfer the knowledge to other regions,’ she said.

As an example, Lara described how the resources had been used by the Global Ghost Gear Initiative, an alliance involving the fishing industry, private sector, corporates, NGOs, academia and governments that focuses on solving the problem of lost and abandoned fishing gear.

‘The Global Ghost Gear Initiative developed a best-practice framework for the management of aquaculture gear, and they used four of our reports, our marine inventory and our toolbox to help build it,’ said Lara.

With a section in the toolbox for people to contribute ideas, she hopes it will grow further and that the knowledge base will ultimately lead to more practical solutions. ‘The idea is that it’s for everyone and fed by everyone,’ she said.

Lara said that promise was shown by AQUA-LIT being invited to present at events in locations such as the Black Sea, and for a Latin American audience, reflecting the significant need for this type of information and its importance as a widespread issue. ‘I think the value of AQUA-LIT is having done that first step,’ she said.

With In-No-Plastic likewise hoping to provide foundations to drive forward solutions to marine waste, the problem of plastics and other litter is set to be tackled from multiple angles.

That will also require wide societal strategies to deal with waste, said Comerford. ‘It’s a holistic approach we need,’ he said. ‘You need to look at everything in the environment currently, but also we can be a bit cleverer about our products in terms of sustainability and end-of-life options.’

This article was originally published in Horizon, the EU Research and Innovation magazine.

Free-ranging Risso’s dolphin (Grampus griseus) swimming with plastic litter © Massimiliano Rosso for Maelstrom H2020 project

By Sandrine Ceurstemont

If you happened to be around the coast of Dubrovnik, Croatia in September 2021, you might have spotted two robots scouring the seafloor for debris. The robots were embarking on their inaugural mission and being tested in a real-world environment for the first time, to gauge their ability to perform certain tasks such as recognising garbage and manoeuvring underwater. ‘We think that our project is the first one that will collect underwater litter in an automatic way with robots,’ said Dr Bart De Schutter, a professor at Delft University of Technology in the Netherlands and coordinator of the SeaClear project.

The robots are an example of new innovations being developed to clean up underwater litter. Oceans are thought to contain between 22 and 66 million tonnes of waste, which can differ in type from area to area, where about 94% of it is located on the seafloor. Fishing equipment discarded by fishermen, such as nets, are prevalent in some coastal areas while plastic and glass bottles are mostly found in others, for example. ‘We also sometimes see construction material (in the water) like blocks of concrete or tyres and car batteries,’ said Dr De Schutter.

When litter enters oceans and seas it can be carried by currents to different parts of the world and even pollute remote areas. Marine animals can be affected if they swallow garbage or are trapped in it while human health is also at risk if tiny pieces end up in our food. ‘It’s a very serious problem that we need to tackle,’ said Dr Fantina Madricardo, a researcher at the Institute of Marine Sciences – National Research Council (ISMAR-CNR) in Venice, Italy and coordinator of the Maelstrom project.

(Our robotic system) will be much more efficient, cost effective and safer than the current solution which is based on human divers.

Human divers are currently deployed to pick up waste in some marine areas but it’s not an ideal solution. Experienced divers are needed, which can be hard to find, while the amount of time they can spend underwater is limited by their air supply. Some areas may also be unsafe for humans, due to contamination for example. ‘These are aspects that the automated system we are developing can overcome,’ said Dr De Schutter. ‘(It) will be much more efficient, cost effective and safer than the current solution which is based on human divers.’

SeaClear’s ROV TORTUGA is known as ‘the cleaner’ robot. It collects the litter from the seafloor. @ SeaClear, 2021

A team of litter-seeking robots

Dr De Schutter and his team are building a prototype of their system for the SeaClear project, which is made up of four different robots that will work collaboratively. A robotic vessel, which remains on the water’s surface, will act as a hub by providing electrical power to the other robots and will contain a computer that is the main brain of the system. The three other robots – two that operate underwater and an aerial drone – will be tethered to the vessel.

The system will be able to distinguish between litter and other items on the seafloor, such as animals and seaweed, by using artificial intelligence. An algorithm will be trained with several images of various items it might encounter, from plastic bottles to fish, so that it learns to tell them apart and identify trash

One underwater robot will be responsible for finding litter by venturing close to the sea floor to take close-up scans using cameras and sonar. The drone will also help search for garbage when the water is clear by flying over an area of interest, while in murky areas it will look out for obstacles to avoid such as ships. The system will be able to distinguish between litter and other items on the seafloor, such as animals and seaweed, by using artificial intelligence. An algorithm will be trained with several images of various items it might encounter, from plastic bottles to fish, so that it learns to tell them apart and identify trash.

Litter collection will be taken care of by the second underwater robot, which will pick up items mapped out by its companions. Equipped with a gripper and a suction device, it will collect pieces of waste and deposit them into a tethered basket placed on the seafloor that will later be brought to the surface. ‘We did some initial tests near Dubrovnik where one plastic bottle was deposited on purpose and we collected it with a gripper robot,’ said Dr De Schutter. ‘We will have more experiments where we will try to recognise more pieces of trash in more difficult circumstances and then collect them with the robot.’

Impact on underwater clean-up

Dr De Schutter and his colleagues think that their system will eventually be able to detect up to 90% of litter on the seafloor and collect about 80% of what it identifies. This is in line with some of the objectives of the EU Mission Restore Our Oceans and Waters by 2030, which is aiming to eliminate pollution and restore marine ecosystems by reducing litter at sea.

When the project is over at the end of 2023, the team expects to sell about ten of their automated systems in the next five to seven years. They think it will be of interest to local governments in coastal regions, especially in touristic areas, while companies may also be interested in buying the system and providing a clean-up service or renting out the robots. ‘These are the two main directions that we are looking at,’ said Dr De Schutter.

Honing in on litter hotspots

Another team is also developing a robotic system to tackle garbage on the seafloor as part of the Maelstrom project. However, their first step is to identify hotspots underwater where litter accumulates so that they will know where it should be deployed. Different factors such as water currents, the speed at which a particular discarded item sinks, and underwater features such as canyons all affect where litter will pool. ‘We are developing a mathematical model that can predict where the litter will end up,’ said Dr Madricardo.

Their robotic system, which is being tested near Venice, is composed of a floating platform with eight cables that are connected to a mobile robot that will move around on the seafloor beneath it to collect waste items in a box, using a gripper, hook or suction device depending on the size of the litter. The position and orientation of the robot can be controlled by adjusting the length and tension of the cables and will initially be operated by a human on the platform. However, using artificial intelligence, the robot will learn to recognise different objects and will eventually be able to function independently.

Repurposing underwater litter

Dr Madricardo and her colleagues are also aiming to recycle all the litter that is picked up. A second robot will be tasked with sorting through the retrieved waste and classifying it based on what it is made of, such as organic material, plastic or textiles. Then, the project is teaming up with industrial partners involved in different types of recycling, from plastic to chemical to fibreglass, to transform what they have recovered.

We want to demonstrate that you can really try to recycle everything, which is not easy

Dirty and mixed waste plastics are difficult to recycle, so the team used a portable pyrolysis plant developed under the earlier marGnet project to turn waste plastic into fuel to power their removal technology. This fits with the EU’s goal to move towards a circular economy, where existing products and materials are repurposed for as long as possible, as part of the European Green Deal and Plastics Strategy. ‘We want to demonstrate that you can really try to recycle everything, which is not easy,’ said Dr Madricardo.

Harnessing bubbles to clean up rivers

Dr Madricardo and her colleagues are also developing a second technology focussed on removing litter floating in rivers so that it can be intercepted before it reaches the sea. A curtain of bubbles, called a Bubble Barrier, will be created by pumping air through a perforated tube placed on the bottom of a river, which produces an upwards current to direct litter towards the surface and eventually to the banks where it is collected.

The system has been tested in canals in the Netherlands and is currently being trialled in a river north of Porto in Portugal, where it is expected to be implemented in June. ‘It’s a simple idea that does not have an impact on (boat) navigation,’ said Dr Madricardo. ‘We believe it will not have a negative impact on fauna either, but we will check that.’

Although new technologies will help tackle underwater litter, Dr Madricardo and her team are also aiming to reduce the amount of waste that ends up in water bodies in the first place. The Maelstrom project therefore involves outreach efforts, such as organised coastal clean-up campaigns, to inform and engage citizens about what they can do to limit marine litter. ‘We really believe that a change (in society) is needed,’ said Dr Madricardo. ‘There are technologies (available) but we also need to make a collective effort to solve this problem.’

The research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

DNA origami is a technique that allows scientists to create 3D bots made from DNA. Image credit – Daniele Adami, licensed under CC BY 2.0

By Anthony King

Doctors know that we need smarter medicines to target the bad guys only. One hope is that tiny robots on the scale of a billionth of a metre can come to the rescue, delivering drugs directly to rogue cancer cells. To make these nanorobots, researchers in Europe are turning to the basic building blocks of life – DNA.

Today robots come in all shapes and sizes. One of the strongest industrial robots can lift cars weighing over two tons. But materials such as silicon are not so suitable at the smallest scales.

While you can make really small patterns in solid silicon, you can’t really make it into mechanical devices below 100 nanometres, says Professor Kurt Gothelf, chemist and DNA nanotechnologist at Aarhus University in Denmark. That’s where DNA comes in. ‘The diameter of the DNA helix is only two nanometres,’ says Prof. Gothelf. A red blood cell is about 6,000 nanometres across.

Lego

Dr Tania Patiño, a nanotechnologist at the University of Rome in Italy, says DNA is like Lego. ‘You have these tiny building blocks and you can put them together to create any shape you want,’ she explained. To continue the analogy, DNA comes in four different coloured blocks and two of the colours pair up opposite one another. This makes them predictable.

Once you string a line of DNA blocks together, another line will pair up opposite. Scientists have learnt how to string DNA together in such a way that they introduce splits and bends. ‘By clever design, you branch out DNA strands so that you now have three dimensions,’ said Prof Gothelf. ‘It is very easy to predict how it folds.’

Dr Patiño is developing self-propelled DNA nanorobotics in her project, DNA-Bots. ‘DNA is highly tuneable,’ she said. ‘We can have software that shows us which sequences produce which shape. This is not possible with other materials at this tiny scale.’

While DNA nanorobots are a long way from being used in people, with Prof. Gothelf saying that ‘we won’t see any medicines based on this in the next ten years,’ progress is being made in the lab. Already scientists can obtain a string of DNA from a virus, and then design using software shorter stretches of DNA to pair with and bend the string into a desired shape. ‘This amazing technique is called DNA origami,’ said Prof. Gothelf. It allows scientists to create 3D bots made from DNA.

In an early breakthrough, Prof. Gothelf’s research lab made a DNA box with a lid that opened. Later, another group built a barrel-shaped robot that could open when it recognised cancer proteins, and release antibody fragments. This strategy is being pursued so that one day a DNA robot might approach a tumour, bind to it and release its killer cargo.

‘With nanorobots we could have more specific delivery to a tumour,’ said Dr Patiño. ‘We don’t want our drugs to be delivered to the whole body.’ She is in the lab of Professor Francesco Ricci, which works on DNA devices for the detection of antibodies and delivery of drugs.

Meanwhile, the network Prof. Gothelf heads up, DNA-Robotics, is training young scientists to make parts for DNA robotics that can perform certain actions. Prof. Gothelf is working on a ‘bolt and cable’ that resembles a handbrake on a bike, where force in one place makes a change in another part of the DNA robot. A critical idea in the network is to ‘plug and play,’ meaning that any parts built will be compatible in a future robot.

This has the potential to make a completely new generation of drugs.

Prof. Kurt Gothelf, Aarhus University, Denmark

Bloodstream

As well as carrying out specific functions, most robots can move. DNA robots are too miniscule to swim against our bloodstream, but it is still possible to engineer into them useful little engines using enzymes.

Dr Patiño previously developed a DNA nanoswitch that could sense the acidity of its environment. Her DNA device also worked as a self-propelling micromotor thanks to an enzyme that reacted with common urease molecules found in our bodies and acted as a power source. ‘The chemical reaction can produce sufficient energy to generate movement,’ said Dr Patiño.

Movement is important to get nanorobots to where they need to be. ‘We could inject these robots in the bladder and they harvest the chemical energy using urease and move,’ said Dr Patiño. In future such movement ‘will help them to treat a tumour or a disease site with more efficiency that passive nanoparticles, which cannot move.’ Recently, Patiño and others reported that nanoparticles fitted with nanomotors spread out more evenly than immobile particles when injected into the bladder of mice.

Rather than swim through blood, nanobots might be able to pass through barriers in our body. Most problems delivering drugs are due to these biological barriers, such as mucosal layers, notes Dr Patiño. The barriers are there to impede germs, but often block drugs. Dr Patiño’s self-propelled DNA robots might change these barriers’ permeability or simply motor on through them.

Stability

Nanoparticles can be expelled from a patient’s bladder, but this option isn’t as easy elsewhere in the body, where biodegradable robots that self-destruct might be necessary. DNA is an ideal material, as it is easily broken down inside of us. But this can also be a downside, as the body might quickly chew up a DNA bot before it gets the job done. Scientists are working on coating or camouflaging DNA and strengthening chemical bonds to boost stability.

One other potential downside is that naked pieces of DNA can be viewed by the immune system as signs of bacterial or viral foes. This may trigger an inflammatory reaction. As yet, no DNA nanobot has ever been injected into a person. Nonetheless, Prof. Gothelf is confident that scientists can get around these problems.

Indeed, stability and immune reaction were obstacles that the developers of mRNA vaccines – which deliver genetic instructions into the body inside a nanoparticle – had to get over. ‘The Moderna and the Pfizer (BioNTech) vaccines (for Covid-19) have a modified oligonucleotide strand that is formulated in a nano-vesicle, so it is close to being a small nanorobot,’ said Prof. Gothelf. He foresees a future where DNA nanorobots deliver drugs to exactly where needed. For example, a drug could be attached to a DNA robot with a special linker that gets cut by an enzyme that is only found inside certain cells, thus ensuring that drug is set free at a precise location.

But DNA robotics is not just for nanomedicine. Prof. Gothelf is mixing organic chemistry with DNA nanobots to transmit light along a wire that is just one molecule in width. This could further miniaturise electronics. DNA bots could assist manufacturing at the smallest scales, because they can place molecules at mind bogglingly tiny but precise distances from one another.

For now though, DNA robotics for medicine is what most scientists dream about. ‘You could make structures that are much more intelligent and much more specific than what is possible today,’ said Prof. Gothelf. ‘This has the potential to make a completely new generation of drugs.’

The research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

This post Foldable, organic and easily broken down: Why DNA is the material of choice for nanorobots was originally published on Horizon: the EU Research & Innovation magazine | European Commission.

By Julianna Photopoulos

Jutting out into the sea, the industrial port area of Nordhavn in Denmark’s capital, Copenhagen, is currently being transformed into a futuristic waterfront city district made up of small islets. It’s billed as Scandinavia’s largest metropolitan development project and, when complete, will have living space for 40,000 people and workspace for another 40,000.

At the moment, Nordhavn is only served by a nearby S-train station and bus stops located near the station. There are no buses or trains running within the development area, although there are plans for an elevated metro line, and parking will be discouraged in the new neighbourhood. This is a great opportunity for autonomous vehicles (AVs) to operate as a new public transport solution, connecting this area more efficiently, says Professor Dimitri Konstantas at the University of Geneva in Switzerland.

‘We believe that AVs will become the new form of transport in Europe,’ he said. ‘We want to prove that autonomous vehicles are a sustainable, viable and environmental solution for urban and suburban public transportation.’

Prof. Konstantas is coordinating a project called AVENUE, which aims to do this in four European cities. In Nordhavn, the team plans to roll out autonomous shuttles on a loop with six stops around the seafront. They hope to have them up and running in two years. But once in place, the Nordhavn plan may provide a glimpse of how AV-based public transportation systems could work in the future.

Prof. Konstantas envisages these eventually becoming an on-demand, door-to-door service, where people can get picked up and go where they want rather than predetermined itineraries and bus stops.

In Nordhavn, AVENUE will test and implement an autonomous ‘mobility cloud’, currently under development, to link the shuttles with existing public transport, such as the nearby train station. An on-demand service will ultimately allow passengers to access the available transport with a single app, says Prof. Konstantas.

Integrating autonomous shuttles into the wider transport system is vital if they are to take off, says Guido Di Pasquale from the International Association of Public Transport (UITP) in Brussels, Belgium.

‘Autonomous vehicles have to be deployed as fleets of shared vehicles, fully integrated and complementing public transport,’ he said. ‘This is the only way we can ensure a sustainable usage of AVs in terms of space occupancy, traffic congestion and the environment.’

Single service

Di Pasquale points to a concept known as Mobility-as-a-Service (MaaS) as a possible model for future transport systems. This model combines both public and private transport. It allows users to create, manage and pay trips as a single service with an online account. For example, Uber, UbiGo in Sweden and Transport for Greater Manchester in the UK are exploring MaaS to enable users to get from one destination to another by combining transport and booking it as one trip, depending on their preferred option based on cost, time and convenience.

Di Pasquale coordinates a project called SHOW, which aims to deploy more than 70 automated vehicles in 21 European cities to assess how they can best be integrated with different wider transport systems and diverse users’ needs. They are testing combinations of AV types, from shuttles to cars and buses, in real-life conditions over the next four years. During this time, he expects the project’s AVs to transport more than 1,500,000 people and 350,000 containers of goods. ‘SHOW will be the biggest ever showcase and living lab for AV fleets,’ he said.

He says that most of the cities involved have tested autonomous last-mile vehicles in the past and are keen to include them in their future sustainable urban mobility plans.

However, rolling out AVs requires overcoming city-specific challenges, such as demonstrating safety.

‘Safety and security risks have restricted the urban use of AVs to dedicated lanes and low speed — typically below 20km/h,’ explained Di Pasquale. ‘This strongly diminishes their usefulness and efficiency, as in most city environments there is a lack of space and a high cost to keep or build such dedicated lanes.’

It could also deter users. ‘For most people, a speed barely faster than walking is not an attractive solution,’ he said.

We want to prove that autonomous vehicles are a sustainable, viable and environmental solution for urban and suburban public transportation.

Prof. Dimitri Konstantas, University of Geneva, Switzerland

Di Pasquale hopes novel technology will make higher speed and mixed traffic more secure, and guarantee fleets operating safely by monitoring and controlling them remotely.

Each city participating in SHOW will use autonomous vehicles in various settings, including mixed and dedicated lanes, at various speeds and types of weather. For safety and regulation reasons, all of them will have a driver present.

The objective is to make the vehicle fully autonomous without the need for a driver as well as optimise the service to encourage people to make the shift from ownership of cars to shared services, according to Di Pasquale. ‘This would also make on-demand and last-mile services sustainable in less densely populated areas or rural areas,’ he said.

Authorisation

But the technical issues of making the vehicle autonomous are only a part of the challenge.

There’s also the issue of who pays for it, says Di Pasquale. ‘AVs require sensors onboard, as well as adaptations to the physical and digital infrastructure to be deployed,’ he explained. ‘Their market deployment would require cities to drastically renew their fleets and infrastructures.’

SHOW’s pilots are scheduled to start in two years from now, as each city has to prepare by obtaining the necessary permits and getting the vehicles and technology ready, says Di Pasquale.

Getting authorisation to operate in cities is one of the biggest hurdles. City laws and regulations differ everywhere, says Prof. Konstantas.

AVENUE is still awaiting city licences to test in Nordhavn, despite a national law being passed on 1 July 2017 allowing for AVs to be tested in public areas. Currently, they have pilots taking place in Lyon, France and Luxembourg. In Geneva, the team has managed to get the required licences and the first worldwide on-demand, AV public transportation service will be rolled out on a 69-bus-stop circuit this summer.

AVENUE’s initial results show that cities need to make substantial investments to deploy AVs and to benefit from this technology. The legal and regulatory framework in Europe will also need to be adapted for smooth deployment of services, says Prof. Konstantas.

Both he and Di Pasquale hope their work can pave the way to convince operators and authorities to invest in fleets across Europe’s cities.

‘Depending on the willingness of public authorities, this can take up to four years until we see real, commercially sustainable AV-based public transportation services on a large scale in Europe,’ said Prof. Konstantas.

The research in this article was funded by the EU.

This post Driverless shuttles: what are we waiting for? was originally published on Horizon: the EU Research & Innovation magazine | European Commission.

An automated wheelchair with an exoskeleton arm is designed to help people with varying forms of disability carry out daily tasks independently. Image credit – AIDE, Universidad Miguel Hernandez

by Julianna Photopoulos

Next-generation wheelchairs could incorporate brain-controlled robotic arms and rentable add-on motors in order to help people with disabilities more easily carry out daily tasks or get around a city.

Professor Nicolás García-Aracil from the Universidad Miguel Hernández (UMH) in Elche, Spain, has developed an automated wheelchair with an exoskeleton robotic arm to use at home, as part of a project called AIDE.

It uses artificial intelligence to extract relevant information from the user, such as their behaviour, intentions and emotional state, and also analyses its environmental surroundings, he says.

The system, which is based on an arm exoskeleton attached to a robotised wheelchair, is designed to help people living with various degrees and forms of disabilities carry out daily functions such as eating, drinking, and washing up, on their own and at home. While the user sits in the wheelchair, they wear the robotised arm to help them grasp objects and bring them close — or as the whole system is connected to the home automation system they can ask the wheelchair to move in a specific direction or go into a particular room.

Its mechanical wheels are made to move in narrow spaces, ideal for home-use, and the system can control the environment remotely – for example, switching lights on and off, using the television or making and answering phone calls. What’s more, it can anticipate the person’s needs.

‘We can train artificially intelligent algorithms to predict what the user wants to do,’ said Prof. García-Aracil. ‘Maybe the user is in the kitchen and wants a drink. The system provides their options (on a monitor) so they can control the exoskeleton to raise the glass and drink.’

Multimodal system

The technology isn’t simple. As well as the exoskeleton robotic arm attached to the robotic wheelchair, the chair has a small monitor and uses various sensors, including two cameras to recognise the environment, voice control, eye-tracking glasses to recognise objects, and sensors that capture brain activity, eye movements and signals from muscles.

Depending on each person’s needs and disabilities, the multiple devices are used accordingly. For example, someone with a severe disability such as a cervical spinal cord injury, who wouldn’t otherwise be able to use voice control, could use the brain activity and eye movement sensors combined.

The user wears a cap on their head, filled with electrodes, to record the brain’s activity which controls the exoskeleton hand’s movement, explains Prof. García-Aracil. So when the user sees themself closing their hand onto an object for example, the exoskeleton arm actually does it for them. This technology is called brain-neural-computer interaction (BNCI), where brain — as well as muscle — activity can be recorded and used to interact with an electronic device.

But the system can sometimes make mistakes so there is an abort signal, says Prof. García-Aracil. ‘We use the horizontal movement of the eye, so when you move your eyes to the right you trigger an action, but when you move your eyes to the left you abort that action,’ he explains.

The AIDE prototype was successfully tested last year by 17 people with disabilities including acquired brain injury (ABI), multiple sclerosis (MS), and spinal cord injury (SCI), at the Cedar Foundation in Belfast, Northern Ireland. Its use was also demonstrated at UMH in Elche, with the user asking to be taken to the cafeteria, then asking for a drink, and drinking it with the help of the exoskeletal arm.

Now more work needs to be carried out to make the system easier to use, cheaper and ready for the market, says Prof. García-Aracil.

But it’s not just new high-tech wheelchairs that can increase the functionality for users. Researchers on the FreeWheel project are developing a way of adding motorised units to existing wheelchairs to improve their utility in urban areas.

‘Different settings have different challenges,’ said project coordinator Ilaria Schiavi at IRIS SRL in Torino, Italy. For example, someone with a wheelchair may struggle to go uphill or downhill without any physical assistance whilst outdoors. But this system could allow people using wheelchairs to have an automated wheelchair experience regardless of whether they are indoors or outdoors, she says.

Rentable

The motorised units would attach to manual wheelchairs people already have in order to help them move around more easily and independently, Schiavi explains. These could either be rented for short periods of time and tailored according to the location — an indoor or outdoor environment — or bought, in which case would be completely personalised to the individual.

The researchers are also developing an app for the user which would include services such as ordering a bespoke device to connect the wheelchair and the unit, booking the unit, controlling it, and planning a journey within urban areas for shopping or sightseeing.

‘You have mobility apps that allow you to book cars, for example. Our app will allow the owner of a wheelchair to firstly subscribe to the service, which would include buying a customised interface to use between their own wheelchair and the motorising unit they have booked,’ said Schiavi.

‘A simple customised interface will allow wheelchair users to motorise their exact device, as it is used by them, at a reasonable cost.’

Customisation is made possible through additive manufacturing (AM) technologies, she says. AM technologies build 3D objects by adding materials, such as metal or plastic, layer-by-layer.

Schiavi and her colleagues are exploring various uses for the motorised units and next year, the team plans to test this system with mobility-impaired people in both Greece and Italy. They hope that, once developed, they will be made available like city bicycles in public spaces such as tourist attractions or shopping centres.

The research in this article was funded by the EU.

Automated, networked truck convoys could save fuel and cut down on driving time. Image credit – MAN Truck & Bus

Automation will increasingly allow vehicles to take over certain aspects of driving. However automated functions are still being fine-tuned, for example, to ensure smooth transitions when switching between the human driver and driverless mode.

Standards also need to be set across different car manufacturers, which is one of the goals of a project called L3Pilot. Although each brand can maintain some unique features, automated functions that help with navigating traffic jams, parking and motorway and urban driving must be programmed to do the same thing.

‘It’s like if you rent a car today, your expectation is that it has a gear shift, it has pedals, it has a steering wheel and so on,’ said project coordinator Aria Etemad from Volkswagen Group Research in Wolfsburg, Germany. ‘The approaches and the interfaces to the human driver are the same.’

To get the same functions from different brands to operate in an identical way, the team is creating a code of practice. This will result in a checklist for developers to run through when creating a self-driving function. ‘It’s like a cookbook for how to design and develop automated driving functions,’ said Etemad.

So far, the project team, which includes representatives from 13 vehicle manufacturers, has been conducting initial tests to make sure their company’s technology works. Cars are equipped with several sensors as well as cameras and computers which need to be properly calibrated to deal with real-world traffic.

The next step is to test the automated functions on public roads to ensure that the vehicles are ready. The tests will begin this month. Volunteer drivers will be chosen from diverse backgrounds, including different ages and genders. ‘We plan, all in all, to have 1,000 drivers using 100 vehicles in 10 different countries,’ Etemad said.

Transition

The technologies being trialled will cover a wide range of situations from overtaking on motorways to driving through urban intersections. Long journeys are also planned to see how people are able to transition back to driving after a long time in automated driving mode.

‘We really want to understand if the way we have designed our systems is the way drivers expect them to behave,’ said Etemad.

The team will also be investigating other aspects of automated driving such as the effect on traffic flow and CO2 emissions. Self-driving features are likely to make driving more efficient due to connectivity between vehicles and infrastructure, for example, although research so far has shown mixed results due to other contributing factors such as more people choosing to drive.

The first automated functions, which should be available in the next year, are likely to be for motorway driving, according to Etemad. Parking functions are likely to come to market next followed by automated urban driving, which is much more complex due to additional elements such as pedestrians and cyclists moving around.

‘There will be a good contribution from the project with results about long-term automated driving and a general understanding of people and functions that will impact how these systems are developed,’ said Etemad.

Automated functions are of interest for trucks too, where networked vehicles driving in a convoy could help save fuel, cut down on driving time or help with traffic flow. Truck platooning involves several trucks linking up through a wireless connection when they are close by so that they can share information and use automated functions to drive together as one. But so far, the concept has mostly been demonstrated in trucks of the same brand and in very few cases, with two brands.

‘Each automotive manufacturer tries it out and develops the technique within their own factory,’ said Dr Marika Hoedemaeker, a senior project manager at the Netherlands Organisation for Applied Scientific Research (TNO) in Helmond.

Truck platooning

Hoedemaeker and her project partners are now trying to break new ground by developing truck platooning that works across different brands as part of a project called ENSEMBLE. ‘Now we’re going to show that we can do this together with all the European truck manufacturers,’ said Dr Hoedemaeker.

The first phase, which the team has just completed, involved coming up with specifications that need to be implemented by all manufacturers. For example, braking and speed keeping will be automated whereas steering won’t be.

They’ve also come up with guidelines for how a convoy will respond when it enters a zone with a new speed limit or passes a toll gate, for example. ‘It’s not only communicating with the platoon but also with the outside world,’ she said.

The team is also keen to gauge the impact that a platoon will have on traffic. They would like to calculate its effect on traffic flow and whether it would in fact reduce congestion on the roads. Driving simulators will also be used to see how other drivers react when they encounter an automated truck convoy. ‘Will it change their traffic behaviour or will they never drive in the right lane anymore? There are lots of questions around this other traffic behaviour as well,’ said Dr Hoedemaeker.

Once specifications have been implemented in the trucks, they will start to test platoons on dedicated grounds then on public roads. Since trucks cross borders quite often, they will have to respect laws in different countries which vary across EU member states. The minimum following distance between vehicles, for example, differs from country to country.

In a final showcase event in May 2021, trucks from seven different brands, such as Daimler and Volvo, will drive together in one or more convoys across national borders, most likely to a key goods transport destination such as a large European port.

Following this deployment on European roads, Hoedemaeker expects the first generation platooning trucks to start being manufactured and sold a year after the project ends in 2021.

Since platoons are being developed worldwide, the standards created during the project could also be adopted more widely.

‘I think there is potential that the rest of the world could say they (Europe) already thought about the standards so we can use these and not do the whole thing over again,’ she said.

The research in this article was funded by the EU.

Cozmo robots and their corresponding tablets are being distributed to participants to take home so that they can interact with them for a week for an experiment being carried out by social robotics professor Emily Cross. Image credit – Ruud Hortensius and Emily Cross

People’s interactions with machines, from robots that throw tantrums when they lose a colour-matching game against a human opponent to the bionic limbs that could give us extra abilities, are not just revealing more about how our brains are wired – they are also altering them.

Emily Cross is a professor of social robotics at the University of Glasgow in Scotland who is examining the nature of human-robot relationships and what they can tell us about human cognition.

She defines social robots as machines designed to engage with humans on a social level – from online chatbots to machines with a physical presence, for example, those that check people into hotel rooms.

According to Prof. Cross, as robots can be programmed to perform and replicate specific behaviours, they make excellent tools for shedding light on how our brains work, unlike humans, whose behaviour varies.

‘The central tenets to my questions are, can we use human-robot interaction to better understand the flexibility and fundamental mechanisms of social cognition and the human brain,’ she said.

Brain imaging shows that a sad, happy or neutral robotic expression will engage the same parts of the brain as a human face with similar expressions.

Through their project called Social Robots, Prof. Cross and her team are using neural decoding techniques to probe the extent to which human feelings towards a robot change depending on how it behaves.

Tantrums

When the robots used in the project lose a game, they alternate between throwing tantrums or appearing dejected. ‘So far, people actually find it really funny when the robot gets angry,’ she said. ‘But people do respond to them quite strongly and that’s really interesting to see.’

Having robots as colleagues has been shown to affect humans in complex ways. Researchers at the University of Washington found that when soldiers used robots in bomb disposal, they developed emotional attachments towards them and felt frustration, anger or sadness if their robot was destroyed.

Prof. Cross says that from an evolutionary perspective, this doesn’t make sense. ‘We care about people and perhaps animals that might help us or hurt us,’ she said. ‘But with machines it’s a bit more of a mystery and understanding how far we can push that (to develop social relationships with machines) is a really, really fascinating question.’

It’s important to understand these dynamics since, as she points out, robots are already working as companions in nursing homes or even as tutors in early childhood education. Home care and education are prime areas of social robotics research, with R&D efforts focusing on adults suffering from dementia and young children.

Ten-hour rule

Typically, studies on such groups observe interactions over a relatively short time-span. They rarely exceed what Prof. Cross describes as a ten-hour rule, beyond which study participants tend to get bored of their robotic toys. But her team is looking at how feelings towards robots evolve over time.

As part of the project, the researchers send a palm-sized Cozmo robot home with study participants and instruct them to interact with it every day for a week by playing games or introducing it to their friends and pets. The participants’ brains are imaged at the start and end of that period to track changes.

‘If we’re going to have robots in our home environment, if they’re going to be in our schools teaching our kids across weeks, if not years, if they’re going to be peoples’ social companions, we want to know a lot more than just what happens after ten hours’ (of exposure),’ she said.

‘We want to know how people’s social bonds and relationships to robots change across many, many more hours.’

With such technologies set to become a bigger part of our future, other studies are investigating how the brain reacts to a different kind of robot – wearable robotic limbs that augment the body, providing extra abilities.

Wearables could have social and healthcare benefits. For instance, a third arm could assist surgeons to carry out procedures more safely rather than relying on human assistants, enable people to complete their household chores much faster or help construction workers.

But even as the technology capabilities develop apace, Dr Tamar Makin, a neuroscientist at University College London, UK, is exploring what it would take for the brain to accept and operate a robotic appendage as part of the body, through a five-year project called Embodied Tech.

Additional thumb

In order to understand how the brain deals with an extra body part, Dr Makin’s team asks participants to wear an additional opposable thumb for a week. Created by a designer named Dani Clode, the thumb is controlled by pressure sensors worn on the big toes.

Product designer Dani Clode created a prosthetic opposable thumb for people to wear as an extra digit. Video credit: Dani Clode

With the additional thumb, the augmented hand almost has the capabilities of two hands, giving people extra capacity to carry out actions. The question is what effect that has on the brain.

The study is still underway but preliminary results indicate that the presence of an extra thumb alters the brain’s internal map of what the biological hand looks like. Scans show that the brain represents the fingers as collapsing onto each other, away from the thumb and index finger.

This mirrors what happens in diseases like dystonia, when the representation of fingers begins to merge – for instance, when musicians use their fingers excessively – and causes cramp-like pain. The same effect could theoretically cause pain in the wearer of an extra thumb.

‘One important interim message we have is that there are potential costs, not just benefits, to using augmentation technology,’ said Dr Makin.

She believes that the newness of human augmentation means there are lots of unanswered questions but it’s vital to explore the challenges of wearable robotics in order to fully realise the promises, such as multitasking or safer working conditions.

‘I feel like we have a responsibility to gain a much better understanding of how having good control of an additional body part is going to change the representation of the body parts you already have.’

The research in this article was funded by the European Research Council.

In trials, the ResiBot robot learned to walk again in less than two minutes after one of its legs was removed. Image credit – Antoine Cully / Sorbonne University

It’s part of a field of work that is building machines that can provide real-time help using only limited data as input. Standard machine-learning algorithms often need to process thousands of possibilities before deciding on a solution, which may be impractical in pressurised scenarios where fast adaptation is critical.

After Japan’s Fukushima nuclear disaster in 2011, for example, robots were sent into the power plant to clear up radioactive debris in conditions far too dangerous for humans. The problem, says robotics researcher Professor Jean-Baptiste Mouret is that the robots kept breaking down or came across hazards that stopped them in their tracks.

As part of the ResiBots initiative, he is designing a lower-cost robot that can last long periods without needing constant human maintenance for breakages and are better at overcoming unexpected obstacles.

The ResiBots team is using what it refers to as micro-data learning algorithms, which can help robots adapt in front of one’s eyes in a similar way to how animals react to problems. An animal will, for example, often find a way to continue moving if they get injured, even if they don’t know exactly what the problem is.

In contrast, most current robots self-diagnose a problem before working out a way to overcome it, says Prof. Mouret, principal investigator at ResiBots and a senior researcher at the Inria research centre in France.

‘We’re trying to shortcut this by finding a way for them to react without necessarily having developed an understanding of what’s wrong,’ he said.

Rather than self-diagnosing, the aim for these robots is to learn in a proactive way by trial and error what alternative actions they can take. This could help them overcome difficulties and stop them from shutting down in situations such as disaster scenarios like Fukushima, said Prof. Mouret.

This may not be full artificial intelligence, but Prof. Mouret points out that having knowledge of everything is not essential for getting a robot to work.

‘We’re not trying to solve everything,’ he said. ‘I’m more interested in how they can adapt – and, in fact, adapting to what’s happening is some of what makes animals intelligent.’

Simulated childhood

In one of the most promising approaches developed in the ResiBots project, the robots have a simulated childhood, in which they learn different ways to move their body using an algorithm that searches ahead of time to collect examples of useful behaviours. 

This means that when seeking a way to move, the robots need to choose from one of about 13,000 behaviours rather than an estimated 10⁴⁷ options that standard algorithms could select from. And the aim is for them to try only a handful of these before finding one that works.

Most of ResiBot’s tests are currently being carried out on a six-legged robot that seeks to find new ways to move after having one or more legs removed. In the latest trials, Prof. Mouret said the robots learned to walk in one to two minutes after one of their legs was taken off, meaning they generally need to test fewer than 10 behaviours before finding one that works.

Test robots can learn to overcome a broken leg in under two minutes. Video credit – Horizon

In total, the researchers are working on half a dozen robots at varying levels of complexity including a child-like humanoid robot known as iCub. Though the much more complex iCub is not yet being used in many trials, the team hopes to do so more over time.

‘Humanoids have the potential of being highly versatile and adapting well to environments designed for humans,’ said Prof. Mouret. ‘For instance, nuclear power plants have doors, levers and ladders that were designed for people.’

There are, however, some big challenges still to overcome, including the fact that a robot needs to be moved back to its starting position once a limb is removed, rather than being able to carry on from the injury site towards the target.

Safety

There are also wider safety issues involving such robots – for example, ensuring that they do not harm earthquake survivors while rescuing them, particularly if the robot is learning by trial and error, said Prof. Mouret.

He believes it will be at least four or five years before such a robot could be used in the field, but is hopeful that the techniques can eventually be employed in all types of robot – not just those for disaster situations, but in the home and other scenarios.

But it’s not just mechanics that can help robots navigate the real world. Robots may also adapt better if they can more strongly connect language to reality. 

Professor Gemma Boleda at the Universitat Pompeu Fabra in Spain, has a background in linguistics and her team is trying to link research in this field to artificial intelligence to help machines better understand the world around them, as part of a project called AMORE.

It’s something that could be useful for making technology such as GPS more intelligent. For example, when driving in a car, the GPS system could specify that you turn right where ‘the big tree’ is, distinguishing it from several other trees.

Prof. Boleda says this has been hard to do in the past because of the difficulty of modelling the way humans link language with reality.

‘In the past, language had largely been represented out of context,’ said Prof. Boleda.

AMORE’s aim is to get computers to understand words and concepts in a real-world context rather than as individual words in isolation, she says. For instance, a robot would learn to connect the phrase ‘this dog’ with an actual dog in the room, representing both the words and the real-world entities.

‘The crux of these models is that they are able to learn their own representations from data,’ she added. ‘Before, researchers had to tell the machine what the world looked like.’

Giving machines a better understanding of the world around them will help them do ‘more with less’ in terms of the amount of data they need and get better at predicting outcomes, Prof. Boleda said.

It could also help with the issue of having enough physical space on devices like mobile phones for the next wave of intelligent applications.

‘I am working with language, but this problem of needing a lot of data is a problem that plagues many other domains of artificial intelligence,’ said Prof. Boleda. ‘So if I develop methods that can do more with less, then these can also be applied elsewhere.’

The research in this article was funded by the EU’s European Research Council.

Better tracking of forest data will make the climate change reporting process easier for countries who want compensation for protecting their carbon stock. Image credit – lubasi, licensed under CC BY-SA 2.0

‘If you can measure the biomass you can measure the carbon and get a number which has value for a country,’ said Pedro Freire da Silva, a satellite and flight system expert at Deimos Engenharia, a Portuguese technology company.

International financial institutions, such as the World Bank and the European Investment Bank, provide developing countries with economic support to keep forests’ carbon stocks intact through the UN REDD+ programme.

The more carbon a developing country can show it keeps in its forests, the more money the government could get, which would give them a greater incentive to protect these lands. But, according to Silva, these countries often lack the tools to determine the exact amount of carbon stored in their forests and that means they could be missing out on funding.

‘If you have a 10% error in your carbon stock (estimation), that can have a financial value,’ he said, adding that it also takes governments a lot of time and energy to collect the relevant data about their forests.

To address these challenges, a project called COREGAL developed automated low-cost drones that map biomass. They put a special sensor on drones that fly over forests and analyse Global Positioning System (GPS) and Galileo satellite signals as they bounce back through a tree canopy, which then reveals the biomass density of an area and, in turn, the carbon stocks.

‘The more leaves you have, the more power (from GPS and Galileo) is lost,’ said Silva who coordinated the project. This means when the drone picks up weaker satellite navigation readings there is more biomass below.

‘If you combine this data with satellite data we get a more accurate map of biomass than either would (alone),’ he added.

Sentinels

The project trialled their drone prototype in Portugal, with Brazil in mind as the target end user as it is on the frontline of global deforestation. According to Brazilian government data, an area about five times to size of London was destroyed between August 2017 and July this year.

COREGAL’s drones could end up enabling countries such as Brazil to access more from climate funds, in turn creating a stronger incentive for governments to protect their forests. Silva also believes the drones could act as a deterrent against illegal logging.

‘If people causing deforestation know that there are (drone) flight campaigns or people going to the field to monitor forests it can demotivate them,’ he said. ‘It is like a sentinel system.’

In the meantime, governments in other developing countries still need the tools to help them fight deforestation. According to Dr Thomas Häusler, a forest and climate expert at GAF, a German earth observation company, the many drivers of deforestation make it very difficult to sustainably manage forests.

‘(Deforestation) differs between regions and even in regions you have different drivers,’ said Dr Häusler. ‘In some countries they (governments) give concessions for timber logging and companies are (then) going to huge (untouched forest) areas to selectively log the highest value trees.’

Logging like this is occurring in Brazil, central Africa and Southeast Asia. When it happens, Dr Häusler says this can cause huge collateral damage because loggers leave behind roads that local populations use to access previously untouched forests which they further convert for agriculture or harvest wood for energy.

Demand for timber and agricultural produce from developed countries can also drive deforestation in developing countries because their governments see the forest as a source of economic development and then allow expansion.

With such social, political and economic dependency, it can be difficult, and expensive, for governments to implement preventative measures. According to Dr Häusler, to protect untouched forests these governments should be compensated for fighting deforestation.

‘To be compensated you need strong (forest) management and observation (tools),’ said Dr Häusler, who is also the coordinator of EOMonDis, a project developing an Earth-observation-based forest monitoring system that aims to support governments.

Domino effect

They combine high-resolution data from the European Sentinel satellites, available every five days through Copernicus, the EU’s Earth observation system, along with data from the North American Landsat-8 satellite.

Automated processing using special algorithms generates detailed maps on the current and past land use and forest situation to identify the carbon-rich forest areas. The project also has access to satellite data going as far back as the 1970s which can be used to determine how much area has been affected by deforestation.

Like COREGAL, using these maps, and the information they contain, a value is put on the most carbon-rich forest areas, meaning countries can access more money from international financial institutions. The project is almost finished and they soon hope to have a commercially viable system for use.

‘The main focus is the climate change reporting process for countries who want compensation in fighting climate change,’ said Dr Häusler. ‘We can support this process by showing the current land-use situation and show the low and high carbon stocks.’

Another potential user of this system is the international food industry that sells products containing commodities linked to deforestation such as palm oil, cocoa, meat and dairy. In response to their contribution, and social pressure, some of these big companies have committed to zero-deforestation in their supply chain.

‘When someone (a company) is declaring land as zero deforestation, or that palm plantations fit into zero deforestation, they have to prove it,’ said Dr Häusler. ’And a significant result (from the project) is we can now prove that.’

Dr Häusler says the system will help civil society and NGOs who want to make sure industry or governments are behaving themselves as well as allow the different groups to make environmentally sound decisions when choosing land for different purposes.

‘We can show everybody – the government, NGO stakeholders, but also the industry – how to better select the areas they want to use.’

The research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

Researchers are using computer simulations to estimate how 11 different species of extinct archosaurs such as the batrachotomus might have moved. Image credit: John Hutchinson

From about 245 to 66 million years ago, dinosaurs roamed the Earth. Although well-preserved skeletons give us a good idea of what they looked like, the way their limbs worked remains a bigger mystery. But computer simulations may soon provide a realistic glimpse into how some species moved and inform work in fields such as robotics, prosthetics and architecture.

John Hutchinson, a professor of evolutionary biomechanics from the Royal Veterinary College in Hertfordshire, UK, and his colleagues are investigating the locomotion of the earliest, small dinosaurs, as part of the five-year-long Dawndinos project which began in 2016.

‘These dinosaurs have been hugely neglected,’ Prof. Hutchinson said. ‘People – including me – have mostly been studying the celebrity dinosaurs like T. rex.’

About 225 million years ago, during the late Triassic period, these small dinosaurs were in the minority, whereas the bigger crocodile-like animals that lived alongside them were more numerous and diverse. Dinosaurs somehow went on to thrive while most other animals from that period became extinct.

Compared to their quadrupedal, heavy-built contemporaries, what stands out about these early dinosaurs is that they had an erect posture and could, at least intermittently, walk on two limbs. One theory is that their style of locomotion gave them a survival edge.

‘The idea of this project is to test that idea,’ Prof. Hutchinson said.

The team has started to develop computer simulations to estimate how 11 different species of extinct archosaurs – the group of animals that includes crocodiles, birds, their relatives and dinosaurs – might have moved. They will focus on five different types of motion: walking, running, turning, jumping and standing.

Simulations

To test whether their simulations are accurate, the researchers plan to give the same treatment to their living relatives – crocodiles and birds – as well. They will then compare the results to actual measurements of motion to determine how good their computer models of extinct animals are.

‘It will be the first time we ground-truth (test with empirical evidence) these methods very rigorously with the best possible data we can get,’ Prof. Hutchinson said.

So far, they’ve modelled the movement of a Mussaurus – an early cousin of giant plant-eating sauropod dinosaurs such as Brontosaurus. The Mussaurus was much smaller and researchers wanted to see whether it moved on four legs like its larger relatives. The first reconstructions of the animal had it on four legs because it had quite big arms, said Prof. Hutchinson.

Using scans of well-preserved fossils from Argentina, they were able to produce new models of its movement. Prof. Hutchinson and his team found that it was in fact bipedal. It couldn’t have walked on four legs since the palms of its front limbs faced inwards and the forearm joints weren’t capable of rotating downwards. Therefore, it wouldn’t have been able to plant its front legs on the ground.

‘It wasn’t until we put the bones together in a 3D environment and tried playing with their movements that it became clear to us that this wasn’t an animal with very mobile arms and hands,’ Prof. Hutchinson said.

After modelling the large forearm of the Mussaurus, the Dawndinos team realised that it could not be used for walking. Video courtesy: John Hutchinson

Robotics

The simulations produced during the project could be useful for zoologists. But they could have less obvious applications too, for example, helping to improve how robots move, according to Prof. Hutchinson.

Accurate models are needed to replicate the motion of animals, which robotics researchers often take inspiration from. Mimicking a crocodile, for example, could be of interest to create a robot that can both swim and walk on land.

Prof. Hutchinson also regularly gets contacted by film and documentary makers who are interested in using his simulations to create realistic animations. ‘It’s hard to make bigger, or unusual, animals move correctly if the physics isn’t right,’ Prof. Hutchinson said.

Understanding the locomotion of the very largest dinosaurs is the aim of a project being undertaken by paleobiology researcher Alexandra Houssaye and her colleagues from France’s National Centre for Scientific Research and the National Museum of Natural History in Paris. Through their Gravibone project, which began last year, they want to pin down the limb bone adaptations that allow large animals to carry a heavy skeleton.

‘We really want to understand what (bone features) are linked to being massive,’ Dr Houssaye said.

Massive

So far, research has shown that the long bones in the limbs of bigger animals are more robust than those of smaller animals. But this general trend has only been superficially observed. The outer and inner bone structures have adapted over time to help support animals’ weight. For example, whereas smaller terrestrial animals have hollow limb bones, massive ones like elephants, rhinos and hippos have connective tissue in the middle.

Among the largest animals and their ancestors there are also other differences. The limb bones of modern rhinos, for example, are short and heavy. But their prehistoric relatives called Indricotherium, the largest land mammal that ever lived, had a less stocky skeleton. ‘It’s interesting to see that the biggest didn’t have the most massive (frame),’ Dr Houssaye said.

The team is studying both living and extinct animals, focussing on elephants, rhinos, hippos, prehistoric mammals and dinosaurs such as sauropods – a group that includes the biggest terrestrial animals of all time.

So far, they have compared the ankle bones of horses, tapirs, rhinos and fossils of rhinos’ ancestors. They found that for animals of the same mass there were differences depending on if they were short and stout or had longer limbs. In less stocky animals, the two ankle bones tended to be more distinct whereas they were more strongly connected in those that were massively built, probably to reinforce the articulation.

‘It’s not only the mass (of the animal) but how the mass is distributed on the body,’ said Dr Houssaye. ‘For us that was interesting.’

3D modelling

Their next step will be to scan different limb bones and analyse their inner structure. They will also use 3D modelling to figure out how much weight different parts of the bones can handle in different spots, for example.

The results from the project could help make more efficient prosthetics for people and animals, Dr Houssaye said. Designers will be able to better understand how different features of limb bones, such as thickness and orientation, relate to their strength, enabling them to create materials that are lighter but more resistant. 

Similarly, Dr Houssaye has also had interest from the construction industry which is looking for new types of materials and more effective building techniques. Pillars supporting heavy buildings, for example, could be made using less material by improving their inner structure instead.

‘How a skeleton adapts (to heavy weight) has implications for construction,’ Dr Houssaye said. ‘(Architects) are trying to create structures that are able to support heavy weight.’

The research in this article was funded by the European Research Council. If you liked this article, please consider sharing it on social media.

]]> https://robohub.org/memory-jogging-robot-to-keep-people-sharp-in-smart-retirement-homes/ Mon, 01 Oct 2018 10:07:08 +0000 https://robohub.org/memory-jogging-robot-to-keep-people-sharp-in-smart-retirement-homes/

Sensors placed throughout a retirement home helped the ENRICHME robot to keep track of the movements and activities of residents taking part in the project’s trial. Image credit – ENRICHME

by Steve Gillman

Almost a fifth of the European population are over 65 years old, but while quality of life for this age bracket is better than ever before, many will at some point suffer from a decline in their mental abilities.

Without adequate care, cognitive abilities can decline quicker, yet with the right support people can live longer, healthier and more independent lives. Researchers working on a project called ENRICHME have attempted to address this by developing a robotic assistant to help improve mental acuity.

‘One of the big problems of mild cognitive impairment is temporary memory loss,’ said Dr Nicola Bellotto from the School of Computer Science at the University of Lincoln in the UK and one of the principal investigators of ENRICHME.

‘The goal of the project was to assist and monitor people with cognitive impairments and offer basic interactions to help a person maintain their cognitive abilities for longer.’

The robot moves around a home providing reminders about medication as well as offering regular physical and mental exercises – it can even keep tabs on items that are easily misplaced.

A trial was conducted with the robot in three retirement homes in England, Greece and Poland. At each location the robot helped one or more residents, and was linked to sensors placed throughout the building to track the movements and activities of those taking part.

‘All this information was used by the robot,’ said Dr Bellotto. ‘If a person was in the bedroom or kitchen the robot could rely on the sensors to know where the person is.’

The robot was also kitted out with a thermal camera so it could measure the temperature of a person in real time, allowing it to estimate the levels of their respiration and heartbeat. This could reveal if someone was experiencing high levels of stress related to a particular activity and inform the robot to act accordingly.

This approach is based around a principle called ambient assisted living, which combines technology and human carers to improve support for older people. In ENRICHME’s case, Dr Bellotto said their robots could be operated by healthcare professionals to provide tailored care for elderly patients.

The users involved in the trials showed high level of engagement with the robot they were living with – even naming it Alfie in one case – and also provided good feedback, said Dr Bellotto. But he added that it is still a few years away from being rolled out to the wider public.

‘Some of the challenges we had were how to approach the right person in these different environments because sometimes a person lives with multiple people, or because the rooms are small and cluttered and it is simply not possible for the robot to move safely from one point to another,’ he said.

Dr Bellotto and his team are now applying for funding for new projects to solve the remaining technical problems, which hopefully one day will help them take the robot one step closer to commercialisation.

This type of solution would help increase people’s physical and psychological wellbeing, which could help to reduce public spending on care for older people. In 2016 the total cost of ageing in the EU was 25% of GDP, and this figure is expected to rise in the coming decades.

Ageing populations

‘One of the big challenges we have in Europe is the high number of elderly people that the public health system has to deal with,’ said Professor María Inés Torres, a computer scientist from the University of the Basque Country in Spain.

Older people can fall into bad habits for a variety of reasons, such as being unable to go for walks and cook healthy meals because of physical limitations. The loss of a loved one can also lead to reduced socialising, exercising or eating well. These unhealthy habits are all exacerbated by depression caused by loneliness, and with 32% of people aged over 65 living alone in Europe, this is a significant challenge to overcome.

‘If you are able to decrease the correlation between depression and age, so keeping people engaged with life in general and social activities, these people aren’t going to visit the doctor as much,’ said Prof. Torres, who is also the coordinator of EMPATHIC, a project that has developed a virtual coach to help assist elderly people to live independently.

The coach will be available on smart devices such as phones or tablets and is focused on engaging with someone to help them keep up the healthier habits they may have had in the past.

‘The main goal is that the user reflects a little bit and then they can agree to try something,’ said Prof. Torres.

For instance, the coach may ask users if they would like go to the local market to prepare their favourite dinner and then turn it into a social activity by inviting a friend to come along. This type of approach addresses the three key areas that cause older people’s health to deteriorate, said Prof. Torres, which are poor nutrition, physical activity and loneliness.

For the coach to be effective the researchers have to build a personal profile for each user, as every person is different and requires specific suggestions relevant to them. They do this by building a database for each person over time and combining it with dialogue designed around the user’s culture.

The researchers are testing the virtual coach on smart devices with 250 older people in three areas – Spain, France and Norway – who are already providing feedback on what works and what doesn’t, which will increase the chances of the virtual coach actually being used.

By the end of the project in 2020, the researchers hope to have a prototype ready for the market, but Prof. Torres insisted that it will not replace healthcare professionals. Instead she sees the smart coach as another tool to help older people live a more independent life – and in doing so reduce the pressure on public healthcare systems.

The research in this article was funded by the EU. If you liked this article, please consider sharing it on social media.

]]> https://robohub.org/we-need-a-drone-register-to-ensure-privacy-and-safety-air-traffic-expert/ Tue, 24 Jul 2018 22:12:54 +0000 https://robohub.org/we-need-a-drone-register-to-ensure-privacy-and-safety-air-traffic-expert/

By Catherine Collins
There is a fine line between the benefits of using drones and possible misuse. Image credit – Pxhere, licensed under CC0

Requiring drones to identify and authorise themselves before they can fly, which could be achieved by fitting them with SIM cards, could help to protect people’s privacy by providing an effective way to register both users and machines, according to air traffic management expert Robin Garrity.

He has been working on the U-space plan, which sets out a vision for how drones can be integrated into airspace, particularly in urban environments. It is part of work being conducted by the SESAR Joint Undertaking, a public-private partnership that coordinates EU research activities in air traffic management.

It is likely that we’ll be seeing more and more drones in our skies, particularly in cities. What do you see as the main benefits of this?

‘Transport of medical supplies, for example. In the press, there are examples of patients who have died because an organ that they were waiting for was stuck in traffic, or something similar. There was a story in Belgium about this. That’s a tragedy that could be averted by the use of drones, where an organ is attached to a drone and can be flown to its destination directly. It’s quick, it’s simple, and it would not be subject to the problems of traffic.

‘The police authorities and fire departments could use drones at accident and crime scenes to collect evidence, or to help manage the incident. Drones could also be used by air accident investigators to help them to oversee a site, or even to try to recreate the final moments of a flight before a crash.

‘An element of this, which is increasing in importance, is urban air mobility (UAM). UAM is a flying taxi, if you like. Some of the vehicle prototypes are already flying – the idea is that you call your taxi and a drone arrives, you step in and off it goes.

‘There is also a growing demand for last-mile package deliveries, which could both speed up delivery of important packages and reduce the number of trucks on the road.’

With all these benefits, what are some of the challenges drones bring?

‘There is a need to maintain privacy as a fundamental principle of U-space services. Those are easy words to say, but the problem comes in implementation. On one end of the scale, you could say people live in cities, therefore don’t fly small drones over cities. That would solve the privacy issue but, of course, then you would kill the market and that would be counterproductive.

‘It’s very difficult sometimes to identify what is an authorised mission and what’s an unauthorised mission. For example, somebody could abuse privacy by flying a drone up the outside of an apartment building to peek into somebody’s window and take pictures – but the same drone could fly up to the top of the same apartment building to inspect a damaged roof or to check if drains are blocked, which would be a very good use of a drone.’

What’s the solution?

‘We envisage producing a register identifying each drone. This could be done through a system similar to the SIM card in a mobile phone, so that when you get your drone, it won’t do anything unless it confirms that it is registered and authorised. You put your ‘SIM card’ in and then the first thing it does is it connects your drone to a service provider, and then it says yes, this is a legitimate user and you are allowed to fly, and that is determined by whatever process is put in place.

‘But if it is an unregistered drone, or if you haven’t recorded your flight in the system, the authorities can still track it. The intention is that the U-space system can track all flying drones and identify their operators, whether or not they are on an authorised mission.’

How will you prevent drones from flying into potentially dangerous areas, like into airspace reserved for commercial aviation?

‘The drone knows where it is due to its navigation capability, usually GPS (Global Positioning System). There’s a digital map that’s already uploaded to the drone, and a comparison is made between its position and the known areas where it can’t fly – a process called geofencing.

‘If its geofencing database isn’t up to date, the drone won’t fly. It will only start to fly if it’s in an area that it knows it’s allowed to fly in. If you live in an area that’s close to Brussels Airport, for example, and you decide you want to fly your drone, you can try and launch it, but it will stay firmly on the ground.

‘In fact, some drones are already fitted with geofencing capability. It’s not mandated, simply because at the moment, there are liability issues. If a drone flies into a no-go zone because the geofencing data is wrong, who’s liable? The capability exists, but how the service should be presented, delivered and implemented is still being determined.’

Drones need to be registered and their use regulated to maintain privacy, says Robin Garrity. Image credit – SESAR Joint Undertaking

What are some other challenges you anticipate with more and more drones in our cities?

‘We’re expecting a much higher degree of automation in vehicles. If someone puts up a tower crane along a drone’s route, the drone would have to notice the tower crane and avoid it, and if there’s another drone coming in its direction, it has to see that and avoid it. Of course, if there is a police helicopter flying around looking at a car crash for example, it has to avoid that too. That is much harder than it seems. A variety of technologies that exist could solve this issue, but particularly where there’s manned aviation involved, the need to guarantee quality and performance makes this a difficult problem to solve.’

What would happen if a drone flew into the path of a medical drone in order to avoid a helicopter?

‘A detect and avoid system could work with the same philosophy as the Traffic Alert and Collision Avoidance System (TCAS) that’s currently used as a safety net to prevent collisions between airliners, although the technology itself may be very different. Under this paradigm, the drone detects the intruder and avoids it. If it then sees another drone, it or the other drone, avoids in turn, and so on. Of course, it’s not quite as simple as that, and research into Detect and Avoid is still very much ongoing. Manned aircraft will always have priority, and some classes of drone could also be given priority status, such as medical drones referred to before. The European Commission has asked SESAR Joint Undertaking to investigate this problem, and we have a number of research projects underway at the moment which are looking at these particular issues.’

What are the next steps for the U-Space initiative?

‘The next steps for us are to complete the up-and-running exploratory research projects, including one project defining the operational concept, and several more looking at technological issues. We are also running three large demonstration projects, with several more about to start, including one dedicated to geofencing. All of these research projects will provide important output and we’ll be working very closely with the European Aviation Safety Authority (EASA) to help them refine the final drone regulations, and with EUROCONTROL to ensure a harmonised approach to achieving safe drone integration.’

State of play in Europe

In 2016, the European Commission proposed the establishment of an automated traffic management system for drones operating at low-level, referred to as “U-space”. 

Up until now, EU Member States have been solely responsible for smaller drones (weighing less than 150 kilograms). However, legislation was adopted on 26 June 2018 to introduce EU-wide regulations for drones.

The new regulatory framework that is being developed by EASA will ensure that flying drones will be safe even as the skies become increasingly busy. A key safety aspect dictates that drones capable of causing significant damage on impact will have to be registered in a new EU-wide database system. Basic principles regarding safety, security, privacy, data protection and environmental protection are outlined under the new legislation, with more detailed regulations to follow later in 2018.

The European Commission has committed EUR €44 million to integrate drones safely into the airspace and to the SESAR R&D programme under its U-space initiative.

]]> https://robohub.org/garbage-collecting-aqua-drones-and-jellyfish-filters-for-cleaner-oceans/ Fri, 25 May 2018 01:55:00 +0000 http://robohub.org/garbage-collecting-aqua-drones-and-jellyfish-filters-for-cleaner-oceans/

An aqua drone developed by the WasteShark project can collect litter in harbors before it gets carried out into the open sea. Image credit – WasteShark

By Catherine Collins

The cost of sea litter in the EU has been estimated at up to €630 million per year. It is mostly composed of plastics, which take hundreds of years to break down in nature, and has the potential to affect human health through the food chain because plastic waste is eaten by the fish that we consume.

‘I’m an accidental environmentalist,’ said Richard Hardiman, who runs a project called WASTESHARK. He says that while walking at his local harbour one day he stopped to watch two men struggle to scoop litter out of the sea using a pool net. Their inefficiency bothered Hardiman, and he set about trying to solve the problem. It was only when he delved deeper into the issue that he realised how damaging marine litter, and plastic in particular, can be, he says.

‘I started exploring where this trash goes – ocean gyres (circular currents), junk gyres, and they’re just full of plastic. I’m very glad that we’re now doing something to lessen the effects,’ he said.

Hardiman developed an unmanned robot, an aqua drone that cruises around urban waters such as harbours, marinas and canals, eating up marine litter like a Roomba of the sea. The waste is collected in a basket which the WasteShark then brings back to shore to be emptied, sorted and recycled.

The design of the autonomous drone is modelled on a whale shark, the ocean’s largest known fish. These giant filter feeders swim around with their mouths open and lazily eat whatever crosses their path.

It’s powered by rechargeable electric batteries, ensuring that it doesn’t pollute the environment through oil spillage or exhaust fumes, and it is relatively silent, avoiding noise pollution. It produces zero carbon emissions and the device moves quite slowly, allowing fish and birds to merely swim away when it gets too close for comfort.

‘We’ve tested it in areas of natural beauty and natural parks where we know it doesn’t harm the wildlife,’ said Hardiman. ‘We’re quite fortunate in that, all our research shows that it doesn’t affect the wildlife around.’

#WasteShark chomping floating plastic! #Turnthetideonoplastic #plasticfree #plasticpollution #Plasticsoup pic.twitter.com/zgnKEutWGX

— RanMarine Technology (@RanMarineTech) January 26, 2018

WasteShark’s autonomous drone is modelled on a whale shark. Credit – RanMarine Technology

WasteShark is one of a number of new inventions designed to tackle the problem of marine litter. A project called CLAIM is developing five different kinds of technology, one of which is a plasma-based tool called a pyrolyser. 

Useful gas

CLAIM’s pyrolyser will use heat treatment to break down marine litter to a useful gas. Plasma is basically ionised gas, capable of reaching very high temperatures of thousands of degrees. Such heat can break chemical bonds between atoms, converting waste into a type of gas called syngas.

The pyrolyser will be mounted onto a boat collecting floating marine litter – mainly large items of plastic which, if left in the sea, will decay into microplastic – so that the gas can then be used as an eco-friendly fuel to power the boat, or to provide energy for heating in ports.

Dr Nikoleta Bellou of the Hellenic Centre for Marine Research, one of the project coordinators of CLAIM, said: ‘We know that we humans are actually the key drivers for polluting our oceans. Unlike organic material, plastic never disappears in nature and it accumulates in the environment, especially in our oceans. It poses a threat not only to the health of our oceans and to the coasts but to humans, and has social, economic and ecological impacts.’

The researchers chose areas in the Mediterranean and Baltic Seas to act as their case studies throughout the project, and will develop models that can tell scientists which areas are most likely to become litter hotspots. A range of factors influence how littered a beach may be – it’s not only affected by litter louts in the surrounding area but also by circulating winds and currents which can carry litter great distances, dumping the waste on some particular beaches rather than others.

CLAIM’s other methods to tackle plastic pollution include a boom – a series of nets criss-crossing a river that catches all the large litter that would otherwise travel to the sea. The nets are then emptied and the waste is collected for treatment with the pyrolyser. There have been problems with booms in the past, when bad weather conditions cause the nets to overload and break, but CLAIM will use automated cameras and other sensors that could alert relevant authorities when the nets are full.

Microplastics

Large plastic pieces that can be scooped out of the water are one thing, but tiny particles known as microplastics that are less than 5mm wide pose a different problem. Scientists on the GoJelly project are using a surprising ingredient to create a filter that prevents microplastics from entering the sea – jellyfish slime.

The filter will be deployed at waste water management plants, a known source of microplastics. The method has already proven to be successful in the lab, and now GoJelly is planning to upscale the biotechnology for industrial use.

Dr Jamileh Javidpour of the GEOMAR Helmholtz Centre for Ocean Research Kiel, who coordinates the project, said: ‘We have to be innovative to stop microplastics from entering the ocean.’

The GoJelly project kills two birds with one stone – tackling the issue of microplastics while simultaneously addressing the problem of jellyfish blooms, where the creatures reproduce in high enough levels to blanket an area of ocean.

Jellyfish are one of the most ancient creatures on the planet, having swum in Earth’s oceans during the time of the dinosaurs. On the whole, due to a decline in natural predators and changes in the environment, they are thriving. When they bloom, jellyfish can attack swimmers and fisheries.

Fishermen often throw caught jellyfish back into the sea as a nuisance but, according to Dr Javidpour, jellyfish can be used much more sustainably. Not only can their slime be used to filter out microplastics, they can also be used as feed for aquaculture, for collagen in anti-ageing products, and even in food.

In fact, part of the GoJelly project involves producing a cookbook, showing people how to make delicious dishes from jellyfish. While Europeans may not be used to cooking with jellyfish, in many Asian cultures they are a daily staple. However, Dr Javidpour stresses that the goal is not to replace normal fisheries.

‘We are mainly ecologists, we know the role of jellyfish as part of a healthy ecosystem,’ she said. ‘We don’t want to switch from classical fishery to jellyfish fishery, but it is part of our task to investigate if it is doable, if it is sustainable.’

The research in this article has been funded by the EU.

]]> https://robohub.org/we-want-to-end-the-de-industrialisation-of-europe-prof-jurgen-ruttgers/ Sun, 20 May 2018 21:55:06 +0000 http://robohub.org/we-want-to-end-the-de-industrialisation-of-europe-prof-jurgen-ruttgers/

Artificial intelligence software technologies are very important for European industry, says Prof. Jürgen Rüttgers. Image credit – Dirk Vorderstraße, licensed under CC BY 2.0

by Rex Merrifield

He leads the High Level Group on Industrial Technologies, which on 24 April released a report called Re-finding industry – Defining Innovation to make recommendations on EU research and innovation priorities for industry in the next funding programme.

Your report says that AI should be designated as a key enabling technology in the next funding programme, which means it is classed as a priority policy area. What does this mean in practice?

‘Artificial intelligence is something relatively new and a field of strong competition, not only in the world, but also in Europe. It is therefore essential to find a common way ahead to encourage our research. Europe has long experience with identifying key enabling technologies and through these we can organise research and innovation, so we find the best answers from new technologies to benefit our industries.

‘That was why we are defining not only a new innovation policy, but also a policy to re-find industry. We want to end the de-industrialisation of Europe of recent years and by setting these technologies as priorities, we can find a strong way ahead for industry.

‘If we do it well, and I am sure that is possible, we also have the possibility in Europe to recover jobs we have been losing abroad. That is an idea that is very closely tied with the idea of strong productivity growth. But we are aiming not only for economic growth. Among our recommendations is that the European Union and Member States aim for inclusive growth and sustainable protection of our planet. That is essential to our idea for the future.’

 

What other technologies are crucial to Europe’s industry?

‘Industry is central to Europe’s economy. It contributes to Europeans’ prosperity and provides jobs to 36 million people in Europe – one in five jobs. So the new key enabling technologies for the 21^st century need to put us in front in the global competition between Europe, the United States and China. We look at these technologies in three areas.

‘It is very important for the EU to prioritise production technologies. These include advanced manufacturing technologies, advanced materials and nanotechnologies, and life science.

‘And in the 21^st century it is essential to encourage digital technologies – micro- and nano-electronics and photonics.

‘Thirdly, we also need to advance cyber technologies and central to this is artificial intelligence, along with cyber security and connectivity.

‘If you look at data generation and handling, big data analytics, machine learning and deep learning, robots and virtual agents, you can see that artificial intelligence software technologies are very important for European industry in this century.’

What impact do you hope that your report will have on research output, industry and for the wider public?

‘Central to the great challenges in the 21^st century is the transition from the industrial society to the knowledge society. In this society, we have a new production factor – knowledge, which is a most important resource.

‘We also face globalisation, through communication without borders. In the future, no economy can be organised only in a national way.

‘The knowledge society will be accelerated by the widespread digitisation of the economy and society, leading to this process being called the ‘digital revolution’. It is comparable in scope and impact to the industrial revolution of some 200 years ago.

‘Many people fear that this change is not good for them, so it is necessary to have open discussion in all European Member States about this great change. If we manage it well, we will have more jobs, more productivity growth and will be at the front of international competition. But we will only have a chance to achieve this future if we have a strong research ecosystem.’

What else needs to happen to encourage innovation?

‘The European policy for the future is research and innovation policy. It is currently under-financed and for a new system of innovation to grow strongly, we must also have a bigger budget. I believe a most important decision for the European Council in coming months is the debate over the budget for 2021-2027.

‘And we also know what is necessary to do over the next decade, together with all citizens in Europe, to achieve success. I believe we also need a clear message to the people of Europe, so that they will agree.’

Your proposals emphasise sustainability, job creation, fighting inequalities and support to democracy. Could you explain the importance of these elements in research and innovation?

‘In research and innovation policy there is not only an argument for jobs, but also the argument that it is good to live in a united Europe with these values.

‘Young people must see they have a chance to work, have good education, to have good vocational or academic jobs. And we need to make sure there is a chance for more new jobs in new start-ups, in new small and medium enterprises, in new industries and in a new innovation system. If we do that, we will see great acceptance among European Union citizens. They will accept new things, not fear them. The reason is that they know we have an inclusive society and not only a very successful economy.’

If you liked this article, please consider sharing it on social media.

]]> https://robohub.org/sherlock-drones-automated-investigators-tackle-toxic-crime-scenes/ Mon, 26 Mar 2018 17:00:45 +0000 http://robohub.org/sherlock-drones-automated-investigators-tackle-toxic-crime-scenes/

Using drones to gather information and samples from a hazardous scene can help incident commanders make critical decisions. Image credit – ROCSAFE

by Anthony King

Crimes that involve chemical, biological, radiological or nuclear (CBRN) materials pose a deadly threat not just to the target of the attack but to innocent bystanders and police investigators. Often, these crimes may involve unusual circumstances or they are terrorist-related incidents, such as an assassination attempt or the sending of poisons through the mail.

In the recent notorious case of poisoning in the UK city of Salisbury in March 2018, a number of first responders and innocent bystanders were treated in hospital after two victims of chemical poisoning were found unconscious on a park bench. One policeman who attended the scene became critically ill after apparent exposure to a suspected chemical weapon, said to be a special nerve agent called novichok. Police said a total of 21 people required medical care after the incident.

Past examples of rare but toxic materials at crime scenes include the 2001 anthrax letter attacks in the US and the Tokyo subway sarin gas attack in 1995. Following the radioactive poisoning of the Russian former spy, Alexander Litvinenko in London, UK, in 2006, investigators detected traces of the toxic radioactive material polonium in many locations around the city.

Despite these dangers, crime scene investigators must begin their forensic investigations immediately. European scientists are developing robot and remote-sensing technology to provide safe ways to assess crime or disaster scenes and begin gathering forensic evidence.

Harm’s way

‘We will send robots into harm’s way instead of humans,’ explained Professor Michael Madden at the National University of Ireland Galway, who coordinates a research project called ROCSAFE. ‘The goal is to improve the safety of crime scene investigators.’

The ROCSAFE project, which ends in 2019, will deploy remote-controlled aerial and ground-based drones equipped with sensors to assess the scene of a CBRN event without exposing investigators to risk. This will help to determine the nature of the threat and gather forensics.

In the first phase of a response, a swarm of drones with cameras will fly into an area to allow investigators to view it remotely. Rugged sensors on the drones will check for potential CBRN hazards. In the second phase, ground-based robots will roll in to collect evidence, such as fingerprint or DNA samples.

The ROCSAFE aerial drone could assess crime or disaster scenes such as that of a derailed train carrying radioactive material. It will deploy sensors for immediate detection of radiation or toxic agents and collect air samples to test later in a laboratory. Meanwhile, a miniature lab-on-a-chip on the drone will screen returned samples for the presence of viruses or bacteria, for instance.

An incident command centre usually receives a huge volume of information in a short space of time – including real-time video and images from the scene. Commanders need to process a lot of confusing information in an extreme situation quickly and so ROCSAFE is also developing smart software to lend a hand.

Rare events

‘These are rare events. This is nobody’s everyday job,’ said Prof. Madden. ‘We want to use artificial intelligence and probabilistic reasoning to reduce cognitive load and draw attention to things that might be of interest.’

As an example, image analysis software might flag an area with damaged vegetation, suggest a possible chemical spill and suggest that samples be taken. Information such as this could be presented to the commander on a screen in the form of a clickable map, in a way that makes their job easier.

Sometimes, the vital evidence itself could be contaminated. ‘There may be some physical evidence we need to collect – a gun or partly exploded material or a liquid sample,’ said Prof. Madden. ‘The robot will pick up, tag and bag the evidence, all in a way that will stand up in court.’ The researchers are constructing a prototype six-wheeled robot for this task that is about 1.5-metres-long and that can handle rough terrain.

Helping forensic teams to deal with hazardous evidence is a new forensic toolbox called GIFT CBRN. The GIFT researchers have devised standard operating procedures on how to handle, package and analyse toxins such as the nerve-agent ricin, which is deadly even in minute amounts.

When the anthrax powder attacks took place in the US in 2001, the initial response of the security services was slow, partly because of the unprecedented situation. GIFT scientists have drawn up ‘how to’ guides for investigators, so they can act quickly in response to an incident.

‘We want to find the bad guys quickly so we can stop them and arrest those involved,’ said Ed van Zalen at the Netherlands Forensic Institute, who coordinated the GIFT project.

Nerve agents

As well as containment, GIFT devised sensing technology such as a battery-powered boxed device that can be brought to a crime scene to identify nerve agents like sarin within an hour or two. This uses electrophoresis, a chemical technique that identifies charged molecules by applying an electric field and analysing their movements. Usually, samples must be collected and returned to the lab for identification, which takes far longer.

Additionally, they developed a camera to detect radioactive material that emits potentially damaging radiation called alpha particles. This form of radiation is extremely difficult to detect and even a Geiger counter – used to detect most radiation – cannot pick it up. The substance polonium, used to murder Litvinenko, emits alpha particles which create radiation poisoning but because it was a novel attack, the failure to detect it initially slowed the police investigation.

‘That detector would have been very helpful at the time of the Litvinenko case,’ said van Zalen.

The research in this article is funded by the EU.

]]> https://robohub.org/swarming-drones-could-help-fight-europes-megafires/ Wed, 21 Mar 2018 16:52:27 +0000 http://robohub.org/swarming-drones-could-help-fight-europes-megafires/

More than 700,000 hectares of land in the EU were destroyed by forest fires between January and September 2017. Image credit – CC0 Public Domain

by Rob Coppinger

Swarms of firefighting drones could one day be deployed to tackle hugely destructive megafires that are becoming increasingly frequent in the Mediterranean region because of climate change, arson and poor landscape management.

It’s one of a number of initiatives looking at how best to fight large fires from the air – a challenge that’s becoming more and more common.

A 2017 report on forest fires by the EU’s Joint Research Centre said that the year would ‘likely be remembered as one of the most devastating wildfire seasons in Europe since records began’, after the destruction of nearly 700,000 hectares of land in the EU by early September.

Such fires are dangerous not only for people who live in the area but also for the crews of people whose job it is to put the fires out. But using intelligent robots to scout the area and drop water can allow humans to stand further back from the danger zone, only looking at the drones’ data to make decisions from the safety of a command and control centre.

Because drones can fly day or night and gain rapid access to previously inaccessible urban or rural fires they can help to save both the lives of the public and first responders.

Torrential

Multiple autonomous drones dropping 600 litres of water every minute during the night while other unmanned vehicles refill to repeat the attack on a raging fire is the vision of Spanish company Drone Hopper. Despite this torrential approach, ‘we are not meant to be competitors with the airplanes and helicopters, we want to be complementary,’ Drone Hopper’s chief executive officer, Pablo Flores, said.

Their drone uses heat cameras to locate the fire, analyse it, send back the data, and identify what type of fire it is. At just over a metre and a half in length, it can be deployed from an aircraft, as well as a ground vehicle.

The drone is like a helicopter and can hover directly over a specific burning area, but it has many propellers. Once over its target it will release its liquid cargo as a mist designed specifically for the fire type identified.

A mist is good at fighting fire because it cools the area by evaporation and it blocks the transfer of heat to anything flammable nearby. To create the right type of mist, the Drone Hopper uses a proprietary magnetic system and the jet wash from its many propellers to direct the released water and nebulise it.

Flores wants to offer his drone, which is still in development, to local authorities for firefighting. ‘They can’t buy a $30 million airplane, but can have this platform and have their own means to (tackle a fire).’ He says that the Drone Hopper UAV costs five times less per litre than a water tanker aircraft.

Spanish company Drone Hopper wants to create a fleet of drones that can drop 600 litres of water every minute during the night. Image credit – Drone Hopper

But, there is a regulatory obstacle. At the moment, it hasn’t been proven that drones can reliably act autonomously, so national rules generally require each one to have a human remote pilot.

Dr Nazim Kemal Ure, an assistant professor in the aerospace department at Istanbul Technical University in Turkey, said: ‘With multiple autonomous systems many things can be achieved much more quickly.’

Swarming

He is developing a way of coordinating drones that he hopes could contribute to a change in regulation. By the end of the year, Dr Ure expects to be field testing autonomous drones and their swarming algorithms, developed under the DUF project.

Like Drone Hopper, ‘we are detecting the fire by using vision,’ Dr Ure explained. In initial testing the image processing will not be done by the drones, but eventually in real-world flight-testing the algorithms will be installed onboard.

His drones would fly over a burning area and, by examining the vegetation and wind direction and other factors, predict the fire’s spread and direction. With that information they would then precisely drop retardant to stop the fire.

Dr Ure added that further flight-testing may see cooperation with the Turkish government’s Ministry of Forestry and involve a controlled fire.

However, there is work yet to be done to improve the computer-generated fire images in the simulated environment they are using for training the artificial intelligence. ‘Our models are, in the graphical parts, not state-of-the-art,’ said Dr Ure. He wants to have ‘hyper-realistic’ fire for the drones’ vision analysis software to learn from. For him, that will help ensure the drones will operate well in the real world.

And in the real world, Dr Ure sees many other applications. ‘I would like to extend this algorithm to other scenarios such as search and rescue and planetary exploration,’ he said.

Types of forest fires
Ground fires occur 25 to 50 cm underground and move slowly, burning through peat and roots. They are notoriously difficult to put out and, if the conditions are right, they can smoulder through the winter and then move above ground in spring.

Surface fires move at a speed of between 3 and 300 metres per minute but burn only the lower vegetation and leave the trees unaffected. Of all the fires, they cause the least damage and are usually easy to put out.

Ladder fires climb up the taller trees and engulf smaller vegetation. Vines and invasive plants help the fire gain momentum.

Crown fires burn trees all the way to the canopy and are the hottest and most dangerous of wildfires. They can spread quickly and are very difficult to put out, partly due to the height of the flames. They can spread beyond natural firebreaks such as rivers through a process called spotting, where wind or hot air carries a piece of burning wood elsewhere and starts a new fire.

The research in this article is funded by the EU. If you liked this article, please consider sharing it on social media.

More info
Drone Hopper
DUF

]]> https://robohub.org/robots-and-workers-of-the-world-unite/ Thu, 15 Feb 2018 22:00:33 +0000 http://robohub.org/robots-and-workers-of-the-world-unite/

Robots in the workforce will give rise to new jobs for humans, including safety engineers, robot specialists and augmented reality experts, according to researchers. Image credit – ‘FANUC robots’, by Mixabest – Own work, CC BY-SA 3.0

Robots are already changing the way we work – particularly in factories – but worries that they will steal our jobs are only part of the picture, as new technologies are also opening up workplace opportunities for workers and are likely to create new jobs in the future.

Last year, the BBC reported that 800 million global workers will lose their jobs to robotic automation by 2030. This statistic, from a McKinsey Global Institute study, led to countless headlines asking, will robots take your job?

The study found that robots will eliminate some jobs, but also create new ones. As the field develops, European roboticists are busy investigating how factory robots could create new opportunities for workers in manufacturing jobs.

The MANUWORK project is collaborating with non-profit group Lantegi Batuak in Spain, which helps to incorporate people with disabilities into the world of work.

The project scientists are testing a number of assistive technologies including augmented reality (AR) displays that can help workers with disabilities in complex tasks such as the assembly and wiring of electrical cabinets. The display shows step-by-step wiring instructions to the worker.

‘We want to introduce a robot to work with them in the assembly process, to indicate cable connections or do some quick quality checks,’ said Dr Kosmas Alexopoulos at the University of Patras in Greece, who coordinates the project. This kind of human-robot collaboration technology could allow greater involvement of people with disabilities in manufacturing roles across Europe.

Predictable

Predictable physical work in factories plays to the strengths of robots, but even the most modern factories are not run entirely by machines.

European car manufacturing is leading the way in adopting robotics. Last year, the number of robots in French car factories rose by 22% to 1,400 units, with around 9 robots for every 100 workers.

Robots are used heavily for the assembly of cars, mainly for body welding and positioning of large metal parts. Later, the skilled human labour comes in to create the interior. ‘Assembling car interiors is complex and it requires the skills of a human to perform,’ explained Dr Sotiris Makris, industrial robotics expert at the University of Patras.

His industrial robots lab in Patras took a leading role in a project called ROBO-PARTNER, which aimed to safely mix people with robots operating in the same workspace to perform a car assembly task. Under normal circumstances, close man-machine teamwork is not possible because robots and humans are kept apart for safety reasons. The human worker brings intelligence and fine skills, while their robot buddy delivers super-human strength and precision.

Allowing robots and humans work together adds flexibility. – Prof. Björn Hein, Karlsruhe Institute of Technology, Germany

The team took a real-life scenario as a starting point. In an automobile factory, the rear axle of a vehicle – which can weigh over 50 kilograms (kgs) – must be brought into position by a worker. They must secure the part and then fit drum brake components, which also involves heavy lifting and manipulation of flexible wire parts.

‘The worker carries the drums, mounts them and must screw them in place. It is quite a stressful process, both physically and cognitively,’ said Dr Makris.

The project developed a robot, capable of lifting 130 kgs, to move the axle and brake parts into place for the human worker. Importantly, the worker remains in the vicinity as the robot – essentially a powerful arm that can move around and grip and lift items – manoeuvres the car parts into position.

Strenuous

At present, one particular 14-kg part for the rear wheel must be lifted 500 times in a single shift by car assembly workers, but the robot could shoulder this strenuous work. This would mean physical strength would no longer be a reason for someone not to do this job.

Video cameras, sensors and adjustable safety zones prevent the robot from moving dangerously close to the worker. The worker interacts with the robot using a control panel, but also uses virtual reality (VR) glasses that visualise the next task for the worker. The completion of each task is signalled using a smart watch and the human remains in charge.

Dr Makris is optimistic that such research will allow people and robots to work side-by-side. There will be new jobs too. Factories of the future will create new roles, such as safety engineers, robot specialists and AR experts, Dr Makris predicted.

Robots are also being used to transform warehouses for online businesses, which are currently divided into two areas. In the centre, wheeled robots shift shelves around and move items to the perimeter, where human workers are waiting. The workers and robots are kept apart for safety. A laser barrier detects any unauthorised entries by people, which will cause the system to shut down. In the case of a robot failure, the warehouse has to be stopped so a service technician can do repairs.

But what if robots and people could safely tango together in warehouses? This is the vision of SafeLog, a project that is developing a flexible warehouse system where humans and automated guided vehicles safely share the same space.

‘With SafeLog, the service technician could just walk in the warehouse and fix the robot, so the warehouse could stay online,’ explained Professor Björn Hein, robotics engineer at the Karlsruhe Institute of Technology, Germany, and coordinator of the project. Right now, there are usually a few dozen robots, so downtime is not a significant issue. As numbers of robots start to increase, downtime could start to be a problem.

Safety vest

SafeLog is creating a safety vest to be worn by warehouse workers. This wirelessly transmits the worker’s location and can be detected by robots nearby, so that they will sense if a human is too close and stop.

For coordination, SafeLog scientists are developing algorithms that track and predict the movements of people using the vest and robots. This would be used by a fleet management system, the silicon brain that will safely guide hundreds of robots around a warehouse.

The workers will wear smart glasses that will warn them about objects close by and help guide them to items in the warehouse.

‘We intend to make it possible for warehouses to become even bigger, because allowing robots and humans work together adds flexibility and makes it easier to extend,’ said Prof. Hein.

The SafeLog system will be put through its paces in a real warehouse in early 2019.

All research in this article is funded by the EU.

]]> https://robohub.org/earworm-melodies-with-strange-aspects-what-happens-when-ai-makes-music/ Fri, 12 Jan 2018 15:57:05 +0000 http://robohub.org/earworm-melodies-with-strange-aspects-what-happens-when-ai-makes-music/

A new AI machine creates new music from songs it’s fed, mimicking their style. Image credit – FlowMachines

by Kevin Casey

The first full-length mainstream music album co-written with the help of artificial intelligence (AI) was released on 12 January and experts believe that the science behind it could lead to a whole new style of music composition.

Popular music has always been fertile ground for technological innovation. From the electric guitar to the studio desk, laptops and the wah-wah pedal, music has the ability to absorb new inventions with ease.

Now, the release of Hello World, the first entire studio album co-created by artists and AI could mark a watershed in music composition.

Stemming from the FlowMachines project, funded by the EU’s European Research Council, the album is the fruits of the labour of 15 artists, music producer Benoit Carré, aka Skygge, and creative software designed by computer scientist and AI expert François Pachet.

Already Belgian pop sensation Stromae and chart-topping Canadian chanteuse Kiesza have been making waves with the single Hello Shadow.

The single Hello Shadow, featuring Stromae and Kiesza, is taken from the AI-co-written album, Hello World. Video credit – SKYGGE MUSIC

The software works by using neural networks – artificial intelligence systems that learn from experience by forming connections over time, thereby mimicking the biological networks of people’s brains. Pachet describes its basic job as ‘to infer the style of a corpus (of music) and generate new things’.

A musician firstly provides ‘inspiration’ to the software by exposing it to a collection of songs. Once the system understands the style required it outputs a new composition.

‘The system … analyses the music in terms of beats, melody and harmony,’ said Pachet, ‘And then outputs an original piece of music based on that style.’

Creative workflow

The design challenge with this software was to make it adapt to the creative workflow of musicians without becoming a nuisance.

‘The core (problem) was how to do that so that (it) takes into account user constraints. Why? Because if you compose music, actually you never do something from scratch from A to Z,’ said Pachet.

He outlines a typical scenario where the AI software generates something and only parts of it are useful but the musician wants to keep it in, drop the rest and generate new sounds using the previous partial output. It’s a complex requirement, in other words.

‘Basically, the main contribution of the project was to find ways to do that, to do that well and to do that fast,’ said Pachet. ‘It was really an algorithmic problem.’ As creative workers driven by intuition, musicians need direct results to maintain their momentum. A clunky tool with ambivalent results would not last long in a creative workflow.

Pachet is satisfied that his technical goal is completed and that the AI will generate music ‘quickly and under user constraints’.

After years of development and refinement, the AI music tool now fits on a laptop, such as to be found in any recording studio, anywhere. In the hands of music producer Carré, the application became the creative tool that built Hello World.

Computer scientist and AI expert Francois Pachet created a system that co-writes music. Image credit – Kevin Casey/ Horizon

Collaboration

As a record producer, Carré collaborated closely with the artists in the studio to write and produce songs. So, as the resident musical expert, can Carré say if this is a new form of music?

‘It’s not a new form of music,’ he said, ‘It’s a new way to create music.’

Carré said he believes the software could lead to a new era in composition. ‘Every time there is a new tool there is a new kind of compositional style. For this project we can see that there is a new kind of melody that was created.’ He describes this as ‘earworm melodies with strange aspects’.

He also says that the process is a real collaboration between human and machine. The system creates original compositions that are then layered into songs in various forms, whether as a beat, a melody or an orchestration. During the process, artists such as Stromae are actively involved in making decisions about what and how to include the muscial fragments the AI provides.

‘You can recognise all the artists because they have made choices that are their identity, I think,’ said Carré.

Pachet concurs. ‘You know in English you say every Lennon needs a McCartney – so that’s the kind of stuff we are aiming at. We are not aiming at autonomous creation. I don’t believe that’s interesting, I don’t believe it’s possible actually, because we have no clue how to give a computer a sense of agency, a sense that something is going somewhere, (that) it has some meaning, a soul, if you want.’

The album’s title Hello World reflects the expression commonly used the very first time someone runs a new computer program or starts a website as proof that is working. Carré believes that Hello World is just the first step and the software signals the start of a whole new way of composing.

‘Maybe not next year, but in five years there will be a new set of tools that helps creators to make music,’ said Carré.

---

More info

FlowMachines

Max Order web comic

]]> https://robohub.org/robotic-bugs-train-insects-to-be-helpers/ Mon, 09 Oct 2017 23:13:14 +0000 http://robohub.org/robotic-bugs-train-insects-to-be-helpers/

Robots help ants with daily chores so they can be accepted into the colony. Image credit – Dr Bertrand Collignon

by Aisling Irwin

Tiny mobile robots are learning to work with insects in the hope the creatures’ sensitive antennae and ability to squeeze into small spaces can be put to use serving humans.

With a soft electronic whirr, a rather unusual looking ant trundles along behind a column of its arthropod comrades as they march off to fetch some food.

While the little insects begin ferrying tiny globules of sugar back home, their mechanical companion bustles forward to effortlessly pick up the entire container and carry it back to the nest.

It is a dramatic demonstration of how robots can be introduced and accepted into insect societies.

But the research, which is being conducted as part of the EU-funded CyBioSys project, could be an important step towards using robots to subtly control, or work alongside, animals or humans.

‘The idea is to be able to solve (a) problem with a better solution than they (the robots and insects) can produce individually,’ said Dr Bertrand Collignon, who is leading the research at the École Polytechnique Fédérale de Lausanne, in Switzerland.

The robots, which ‘live’ with the ants, pick up signs that food has been discovered through a camera mounted inside the nest. The camera alerts the robots when it detects an increasing numbers of ants are departing – a sign that food has been found.

The robots – reprogrammed off-the-shelf Thymio bots managed by simple Raspberry Pi computers – then use sensors to follow the columns of exiting ants. Once the ants have led their robotic counterparts to their discovery, the robots take over, using their superior muscle power to lug it home.

Dr Collignon described this as a ‘cyber-biological system’, which improves both on the natural order, and on what robots could achieve on their own. By getting ants and robots to collaborate, each community plays to its strengths, he says.

‘The ants are good at exploring the environment very efficiently, with many scouts patrolling the vicinity of the nest at the same time,’ said Dr Collignon, who is a Marie Skłodowska-Curie action fellow. ‘But individual ants are not able to transport large amounts of food and some can get lost between the food and the nest.’

‘By getting ants and robots to collaborate, each community plays to its strengths.’

Dr Bertrand Collignon, École Polytechnique Fédérale de Lausanne, Switzerland

Robots are like pack animals in comparison, carrying an order of magnitude more food than an ant can, and accomplishing in a few minutes what would have taken the ants hours.

Dr Collignon believes it is the first project to consider an insect swarm as a biosensor and then embed in a robot the ability to extract data from the colony.

But he also believes this research could be combined with other work teaching robots to communicate with animals. Instead of relying on top-down instructions — like a shepherd dog herding sheep — this would work by subtly influencing them from a position as one of the group.

As many social insects such as ants and bees can form aggressive colonies that normally do not respond well to outsiders, influencing them from within may offer a new approach.

In a previous EU-funded project, LEURRE, a team pioneered the creation of small mobile robots that could interact with cockroaches and influence their collective behaviour.

When kept in a pen together, cockroaches will gradually gather under the same dark shelter. They achieve this simply by following two rules: stay close to other cockroaches, and head for somewhere dark.

But when the researchers released small robots into the pen programmed with slightly different rules — stay close to other cockroaches but prefer a lighter refuge — in time the cockroaches, along with the robots, gathered in the lighter shelter instead.

Dr Collignon believes that the two types of robotic work – collaboration and communication – could find applications in search and rescue, exploring environments too dangerous or inaccessible for humans. Eventually, small animals could be used to get into restricted environments such as collapsed buildings.

By integrating artificial systems, such as robots, into more natural ones – such as a warehouse full of chickens – it could lead to new solutions to help control animal behaviour on farms. An example might be preventing deadly mass panic attacks amongst intensively reared animals by using robots that can detect the early signs of an impending stampede and diverting one by behaving in a different way.

‘The first step is to be able to track what natural agents are doing and react appropriately to that,’ he said. ‘That’s already a tricky thing. Once you have sensed what nature is doing, you can then interact. The robotic agent can do what it has been designed for and then act on the system.’

---

More info
CyBioSys

]]> https://robohub.org/eus-future-cyber-farms-to-utilise-drones-robots-and-sensors/ Fri, 25 Aug 2017 06:52:20 +0000 http://robohub.org/eus-future-cyber-farms-to-utilise-drones-robots-and-sensors/

Farmers could protect the environment and cut down on fertiliser use with swarms of drones. Image credit – ‘Aerial View – Landschaft Markgräflerland’ by Taxiarchos228 is licenced under CC 3.0 unported

by Anthony King

Bee-based maths is helping teach swarms of drones to find weeds, while robotic mowers keep hedgerows in shape.

‘We observe the behaviour of bees. We gain knowledge of how the bees solve problems and with this we obtain rules of interaction that can be adapted to tell us how the robot swarms should work together,’ said Vito Trianni at the Institute of Cognitive Sciences and Technologies of the Italian National Research Council.

Honeybees, for example, run on an algorithm to allow them to choose the best nest site, even though no bee knows the full picture.

Trianni runs an EU-funded research project known as SAGA, which is using the power of robotic groupthink to keep crops weed free.

‘We can use low-cost robots and low-cost cameras. They can even be prone to error, but thanks to the cooperation they will be able to generate precise maps at centimetre scales,’ said Trianni.

‘They will initially spread over the field to inspect it at low resolution, but will then decide on areas that require more focus,’ said Trianni. ‘They can gather together in small groups closer to the ground.’

Importantly the drones make these decisions themselves, as a group.

Next spring, a swarm of the quadcopters will be released over a sugar beet field. They will stay in radio contact with each other and use algorithms learnt from the bees to cooperate and put together a map of weeds. This will then allow for targeted spraying of weeds or their mechanical removal on organic farms.

Today the most common way to control weeds is to spray entire fields with herbicide chemicals. Smarter spraying will save farmers money, but it will also lower the risk of resistance developing to the agrichemicals. And there will be an environmental benefit from spraying less herbicides.

Co-ops

Swarms of drones for mapping crop fields offer a service to farmers, while farm co-ops could even buy swarms themselves.

‘There is no need to fly them every day over your field, so it is possible to share the technology between multiple farmers,’ said Trianni. A co-op might buy 20 to 30 drones, but adjust the size of the swarm to the farm.

The drones are 1.5 kilos in weight and fly for around 20-30 minutes. For large fields, the drone swarms could operate in relay teams, with drones landing and being replaced by others.

It’s the kind of technology that is ideally suited to today’s large-scale farms, as is another remote technology that combines on-the-ground sensor information with satellite data to tell farmers how much nitrogen or water their fields need.

Wheat harvested from a field in Boigneville, 100 km south of Paris, France, in August this year will have been grown with the benefit of this data, as part of pilot being run by an EU-funded project known as IOF2020, which involves over 70 partners and around 200 researchers.

‘Sensors are costing less and less, so at the end of the project we hope to have something farmers or farm cooperatives can deploy in their fields,’ explained Florence Leprince, a plant scientist at Arvalis – Institut du végétal, the French arable farming institute which is running the wheat experiment.

‘This will allow farmers be more precise and not overuse nitrogen or water.’ Florence Leprince, Arvalis – Institut du végétal, France

Adding too much nitrogen to a crop field costs farmers money, but it also has a negative environmental impact. Surplus nitrogen leaches from soils and into rivers and lakes, causing pollution.

It’s needed because satellite pictures can indicate how much nitrogen is in a crop, but not in soil. The sensors will help add detail, though in a way that farmers will find easy to use.

It’s a similar story for the robotic hedge trimmer being developed by a separate group of researchers. All the farmer or groundskeeper needs to do is mark which hedge needs trimming.

‘The user will sketch the garden, though not too accurately,’ said Bob Fisher, computer vision scientist at Edinburgh University, UK, and coordinator of the EU-funded TrimBot2020 project. ‘The robot will go into the garden and come back with a tidied-up sketch map. At that point, the user can say go trim that hedge, or mark what’s needed on the map.’

This autumn will see the arm and the robot base assembled together, while the self-driving bot will be set off around the garden next spring.

More info:
SAGA (part of ECHORD Plus Plus)
IOF2020
TrimBot2020

]]> https://robohub.org/digital-symbiosis-lets-robot-co-workers-predict-human-behaviour/ Fri, 18 Aug 2017 12:55:54 +0000 http://robohub.org/digital-symbiosis-lets-robot-co-workers-predict-human-behaviour/

Robot co-workers could help out with repetitive jobs and heavy lifting by reacting to human actions. Image credit – Italian Institute of Technology

by Anthony King
Stephen Hawking and Elon Musk fear that the robotic revolution may already be underway, but automation isn’t going to take over just yet – first machines will work alongside us.

Robots across the world help out in factories by taking on heavy lifting or repetitive jobs, but the walking, talking kind may soon collaborate with people, thanks to European robotics researchers building prototypes that anticipate human actions.

‘Ideally robots should be able to sense interactional forces, like carrying a table with someone,’ said Francesco Nori, who coordinates the EU-funded An.Dy project which aims to advance human-robot collaboration. ‘(Robots) need to know what the human is about to do and what they can do to help.’

In any coordinated activity, whether dancing or lifting a table together, timing is crucial and that means a robot needs to anticipate before a person acts.

‘Today, robots just react – half a second of anticipation might be enough,’ said Nori, who works at the Italian Institute of Technology which is renowned for its humanoid robot called iCub, that will be educated in human behaviour from data collected during the An.Dy project.

The data will flow from a special high-tech suit that lies at the heart of the project – the AndySuit. This tight suit is studded with sensors to track movement, acceleration of limbs and muscle power as a person performs actions alone or in combination with a humanoid robot.

A special high-tech suit known as the AndySuit allows a person to perform actions alongside a robot. Image credit – Italian Institute of Technology

This sends data to a robot similar to iCub so that it can recognise what the human is doing and predict the next action just ahead of time. The collaborative robot – also known as a cobot – would then be programmed to support the worker.

‘The robot would recognise a good posture and a bad posture and would work so that it gives you an object in the right way to avoid injury,’ explained Nori, adding that the cobot would adapt its own actions to maximise the comfort of the human.

The robot’s capabilities will come from its library of pre-programmed models of human movement, but also from internal sensors and a mobile phone app. Special sensors that communicate with the iCub are also being developed for the AndySuit, but at the moment it is more appropriate for the robotics lab rather than a factory floor.

To get the robot and AndySuit closer to commercialisation it will be tested in three different scenarios. First, in a workspace where a person works beside a cobot. Second, when a person wears an exoskeleton, which could be useful for workers who must lift heavy loads and can be assisted by a robust metal skeleton around them.

A third scenario will be where a humanoid robot offers assistance and could take turns performing tasks. In this situation, the robot would look like the archetype sci-fi robot; like Sonny from the film iRobot.

Silicon sidekick

A different project will see a human-like prototype robot reach out a helping hand to support technicians, under an EU-funded project called SecondHands led by Ocado Technology in the UK.

‘Ask it to pass the screw driver, and it will respond asking whether you meant the one on the table or in the toolbox.’ Duncan Russel, Ocado Technology

Ocado runs giant automated warehouses that fulfil grocery orders. Its warehouse in Hatfield, north of London, is the size of several football fields and must be temporarily shut down for regular maintenance.

Duncan Russell, research coordinator at Ocado Technology explained: ‘Parts need to be cleaned and parts need replacing. The robot system is being designed to help the technicians with those tasks.’

While the technician stands on a ladder, a robot below would watch what they are doing and provide the next tool or piece of equipment when asked.

‘The robot will understand instructions in regular language – it will be cleverer than you might expect,’ said Russell. ‘Ask it to pass the screw driver, and it will respond asking whether you meant the one on the table or in the toolbox.’

The robot will feature capabilities straight from the Inspector Gadget cartoon series. An extendable torso will allow it to move upwards and telescopic limbs will give it a three metre plus reach.

‘The arm span is 3.1 metres and the torso is around 1.8 metres, which gives it a dynamic reach. This will allow it to offer assistance to technicians up on a ladder,’ said Russell.

This futuristic scenario is being brought to reality by research partners around Europe. Robotics experts at Karlsruhe Institute of Technology in Germany have built a wheeled prototype robot. The plan is for a bipedal robot to be tested in the Ocado robots lab in Hatfield, and for it to be transferred to the warehouse floor for a stint with a real technician.

Karlsruhe is also involved in teaching the robot natural language and together with the Swiss Federal Institute of Technology in Lausanne it is developing a grasping hand, so the helper robot can wield tools with care. The visions system of this silicon sidekick is being developed by researchers at University College London, UK.

The handy robotic helper could also do cross-checking for the maintenance person, perhaps offering a reminder if a particular step is missed, for example.

‘The technician will get more done and faster, so that the shutdown times for maintenance can be shortened,’ said Russell.

More info:
An.Dy
SecondHands

]]> https://robohub.org/artificial-skin-could-allow-robots-to-feel-like-we-do/ Wed, 02 Aug 2017 21:28:45 +0000 http://robohub.org/artificial-skin-could-allow-robots-to-feel-like-we-do/

Researchers are running tests on pig skin to better understand how skin behaves and pave the way for bioengineering applications. Image credit – Dr Aisling Ni Annaidh at University College Dublin

Artificial skin with post-human sensing capabilities, and a better understanding of skin tissue, could pave the way for robots that can feel, smart-transplants and even cyborgs.

Few people would immediately recognise the skin as our bodies’ largest organ, but the adult human has on average two square metres of it. It’s also one of the most important organs and is full of nerve endings that provide us with instant reports of temperature, pressure and pain.

So far the best attempts to copy this remarkable organ have resulted in experimental skin with sensor arrays that, at best, can only measure one particular stimulus.

But the SmartCore project, funded by the EU’s European Research Council and at the Graz University of Technology (TU Graz) in Austria, hopes to create a material that responds to multiple stimuli. To do so requires working at a nanoscale — where one nanometre represents a billionth of a metre — creating embedded arrays of minuscule sensors that could be 2 000 times more sensitive than human skin.

Principal investigator Dr Anna Maria Coclite, an assistant professor at TU Graz’s Institute for Solid State Physics, says the project aims to create a nanoscale sensor which can pick up temperature, humidity and pressure — not separately, but as an all-in-one package.

‘They will be made of a smart polymer core which expands depending on the humidity and temperature, and a piezoelectric shell, which produces an electric current when pressure is applied,’ she said.

These smart cores would be sandwiched between two similarly tiny nanoscale grids of electrodes which sense the electrical charges given off when the sensors ‘feel’ and then transmit this data.

If the team can surmount the primary challenge of distinguishing between the different senses, the first prototype should be ready in 2019, opening the door for a range of test uses.

Robots

Dr Coclite says the first applications of a successful prototype would be in robotics since the artificial skin they’re developing has little in common with our fleshy exterior apart from its ability to sense.

‘The idea is that it could be used in ways, like robotic hands, that are able to sense temperatures,’ said Dr Coclite. ‘Or even things that can be sensed on even a much smaller scale than humans can feel, i.e, robotic hands covered in such an artificial skin material that is able to sense bacteria.’

Moreover, she says the polymers used to create smart cores are so flexible that a successful sensor could potentially be modified in the future to sense other things like the acidity of sweat, which could be integrated into smart clothes that monitor your health while you’re working out.

And perhaps, one day, those who have lost a limb or suffered burns could also benefit from such multi-stimuli sensing capabilities in the form of a convincingly human artificial skin.

‘It would be fantastic if we could apply it to humans, but there’s still lots of work that needs to be done by scientists in turning electronic pulses into signals that could be sent to the brain and recognised,’ said Dr Coclite.

She also says that even once a successful prototype is developed, possible cyborg use in humans would be at least a decade away — especially taking into account the need to test for things like toxicity and how human bodies might accept or reject such materials.

Getting a grip

But before any such solutions are possible, we must learn more about biological tissue mechanics, says Professor Michel Destrade, host scientist of the EU-backed SOFT-TISSUES project, funded by the EU’s Marie Skłodowska-Curie actions.

Prof. Destrade, an applied mathematician at the National University of Ireland Galway, is supporting Marie Skłodowska-Curie fellow Dr Valentina Balbi in developing mathematical models that explain how soft tissue like eyes, brains and skin behave.

‘For example, skin has some very marked mechanical properties,’ said Prof. Destrade. ‘In particular its stretch in the body — sometimes you get a very small cut and it opens up like a ripe fruit.’

This is something he has previously researched with acoustic testing, which uses non-destructive sound waves to investigate tissue structure, instead of chopping up organs for experimentation.

And in SOFT-TISSUES’ skin research, the team hopes to use sound waves and modelling as a cheap and immediate means of finding the tension of skin at any given part of the body for any given person.

‘This is really important to surgeons, who need to know in which direction they should cut skin to avoid extensive scarring,’ explained Prof. Destrade. ‘But also for the people creating artificial skin to know how to deal with mismatches in tension when they connect it to real skin.

‘If you are someone looking to create artificial skin and stretch it onto the body, then you need to know which is the best way to cut and stretch it, the direction of the fibres needed to support it and so on.‘

Dr Balbi reports that the biomedical industry has a real hunger for knowledge provided by mathematical modelling of soft tissues — and especially for use in bioengineering.

She says such knowledge could be useful in areas like cancer research into brain tumour growth and could even help improve the structure of lab-grown human skin as an alternative to donor grafts.

More info:
SmartCore
SOFT-TISSUES

]]> https://robohub.org/swarms-of-smart-drones-to-revolutionise-how-we-watch-sports/ Tue, 11 Jul 2017 12:37:32 +0000 http://robohub.org/swarms-of-smart-drones-to-revolutionise-how-we-watch-sports/

Credit: Flickr/ Ville Hyvönen

by Joe Dodgshun
Drone innovators are transforming the way we watch events, from football matches and boat races to music festivals.

Anyone who has watched coverage of a festival or sports event in the last few years will probably have witnessed commercial drone use — in the form of breathtaking aerial footage.

But a collaboration of universities, research institutes and broadcasters is looking to take this to the next level by using a small swarm of intelligent drones.

The EU-funded MULTIDRONE project seeks to create teams of three to five semi-automated drones that can react to and capture unfolding action at large-scale sports events.

Project coordinator Professor Ioannis Pitas, of the University of Bristol, UK, says the collaboration aims to have prototypes ready for testing by its media partners Deutsche Welle and Rai – Radiotelevisione Italiana within 18 months.

‘Deutsche Welle has two potential uses lined up – filming the Rund um Wannsee boat race in Berlin, Germany, and also filming football matches with drones instead of normal cameras – while Rai is interested in covering cycling races,’ said Prof. Pitas.

‘We think we have the potential to offer a much better film experience at a reduced cost compared to helicopters or single drones, producing a new genre in drone cinematography.’

‘We have the potential to offer a much better film experience at a reduced cost compared to helicopters or single drones, producing a new genre in drone cinematography.’

Professor Ioannis Pitas, University of Bristol, UK

But before they can chase the leader of the Tour de France, MULTIDRONE faces the hefty challenge of creating AI that allows its drones to safely carry out a mission as a team.

Prof. Pitas says safety is the utmost priority, so the drones will include advanced crowd avoidance mechanisms and the ability to make emergency landings.

And it’s not just safety in the case of bad weather, a flat battery or a rogue football.

‘Security of communications is important as a drone could otherwise be hijacked, not just undermining privacy but also raising the possibility that it could be used as a weapon,’ said Prof. Pitas.

The early project phase will have a strong focus on ethics to prevent any issues around privacy.

‘People are sensitive about drones and about being filmed and we’re approaching this in three ways — trying to avoid shooting over private spaces, getting consent from the athletes being followed, and creating mechanisms that decide which persons to follow and blur other faces.’

If they can pull it off, he predicts a huge boost for the European entertainment industry and believes it could lead to much larger drone swarms capable of covering city-wide events.

Drones-on-demand
According to Gartner research, sales of commercial-use drones are set to jump from 110 000 units in 2016 to 174 000 this year. Although 2 million toy drones were snapped up last year for USD 1.7 billion, the commercial market dwarfed this at USD 2.8 billion.

Aside from pure footage, drones have also proven their worth in research, disaster response, construction and even in monitoring industrial assets.

One company trying to open up the market to those needing a sky-high helping hand is Integra Aerial Services, a young drones-as-a-service company.

An offshoot of Danish aeronautics firm Integra Holding Group, INAS, was launched in 2014 thanks to an EU-backed feasibility study.

INAS has more than 25 years of experience in aviation and used its knowledge of the sector’s legislation to shape a business model targeting heavier, more versatile drones weighing up to 25 kilogrammes. And they have already been granted a commercial drone operating license by the Danish Civil Aviation Authority.

These bigger drones have far more endurance than typical toy drones, which can weigh anywhere from 250 grams to several kilos. INAS CEO Gilles Fartek says their bigger size means they can carry multiple sensors, thus collecting all the needed data in one fell swoop, instead of across multiple flights.

For example, one of their drones flies LIDAR (Light Detection and Ranging) radar over Greenland to measure ice thickness as a measure of climate change, but could also carry a 100 megapixel, high-definition camera.

While INAS spends most of the Arctic summer running experiments from the remote host Station Nord in Greenland, Fartek says they’re free to use the drones for different projects in other seasons, mostly in areas of environmental research, mapping and agricultural monitoring.

‘You can’t match the quality of data for the price, but drone-use regulations in Europe are still quite complicated and make between-country operations almost impossible,’ said Fartek.

‘The paradox is that you have an increasing demand for such civil applications across Europe and even in institutional areas like civil protection and maritime safety where they cannot use military drones.’

A single European sky
These issues, and more, should soon be addressed by SESAR, the project which coordinates all EU research and development activities in air traffic management. SESAR plans to deploy a harmonised approach to European airspace management by 2030 in order to meet a predicted leap in air traffic.

Recently SESAR unveiled its blueprint outlining how it plans to make drone use in low-level airspace safe, secure and environmentally friendly. They hope this plan will be ready by 2019, paving the way for an EU drone services market by safely integrating highly automated or autonomous drones into low-level airspace of up to 150 metres.

Modelled after manned aviation traffic management, the plan will include registration of drones and operators, provide information for autonomous drone flights and introduce geo-fencing to limit areas where drones can fly.

The Issue
Emerging drone sectors range from delivery services, collecting industry data, infrastructure inspections, precision agriculture, transportation and logistics.

The market for drone services is expected to grow substantially in the coming years with an estimated worth of EUR 10 billion by 2035.

To support high-potential small- and medium-sized enterprises (SMEs), the European Commission has allocated EUR 3 billion over the period 2014-2020. A further EUR 17 billion was set aside under the Industrial Leadership pillar of the EU’s current research funding programme Horizon 2020.

More info
MULTIDRONE

INAS

A new robot under development can send information on the stiffness, look and feel of a patient to a doctor located kilometres away. Image credit: Accrea

A robotic doctor that can be controlled hundreds of kilometres away by a human counterpart is gearing up for action. Getting a check-up from a robot may sound like something from a sci-fi film, but scientists are closing in on this real-life scenario and have already tested a prototype.

‘The robot at the remote site has different force, humidity and temperature sensors, all capturing information that a doctor would get when they are directly palpating (physically examining) a patient,’ explains Professor Angelika Peer, a robotics researcher at the University of the West of England, UK.

Prof. Peer is also the project coordinator of the EU-funded ReMeDi project, which is developing the robotic doctor to allow medical professionals to examine patients over huge distances.

Through a specially designed surface mounted on a robotic arm, stiffness data of the patient’s abdomen is displayed to the human, allowing the doctor to feel what the remote robot feels. This is made possible thanks to a tool called a haptic device, which has a soft surface reminiscent of skin that can recreate the sense of touch through force and changing its shape.

During the examination, the doctor sits at a desk facing three screens, one showing the doctor’s hand on the faraway patient and a second for teleconferencing with the patient, which will remain an essential part of the exchange.

The third screen displays a special capability of the robot doctor – ultrasonography. This is a medical technique that sends sound pulses into a patient’s body to create a window into the patient. It reveals areas of different densities in the body and is often used to examine pregnant women.

Ultrasonography is also important for flagging injuries or disease in organs such as the heart, liver, kidneys or spleen and can find indications for some types of cancer, too.

‘The system allows a doctor from a remote location to do a first assessment of a patient and make a decision about what should be done, whether to transfer them to hospital or undergo certain treatments,’ said Prof. Peer.

The robot currently resides in a hospital in Poland but scientists have shown the prototype at medical conferences around the world. And they have already been approached by doctors from Australia and Canada where it can take several hours to transfer rural patients to a doctor’s office or hospital.

With the help of a robot, a doctor can talk to a patient, manoeuvre robotic arms, feel what the robot senses and get ultrasounds. Image credit: ReMeDi

‘This is to support an initial diagnosis. The human is still in the loop, but this allows them to perform an examination remotely,’ said Prof. Peer.

Telemedicine

The ReMeDi project could speed up a medical exam and save time for patients and clinics. Another EU-funded project – United4Health (U4H) – looks at a different technology that could be used to remotely diagnose or treat people.

‘We need to transform how we deliver health and care,’ said Professor George Crooks, director of the Scottish Centre for Telehealth & Telecare, UK, which provides services via telephone, web and digital television and coordinates U4H.

This approach is crucial as Europe faces an ageing population and a rise in long-term health conditions like diabetes and heart disease. Telemedicine empowers these types of patients to take steps to help themselves at home, while staying in touch with medical experts via technology. Previous studies showed those with heart failure can be successfully treated this way.

These patients were given equipment to monitor their vital signs and send data back to a hospital. A trial in the UK comparing this self-care group to the standard-care group showed a reduction in mortality, hospital admissions and bed days, says Prof. Crooks.

A similar result was shown in the demonstration sites of the U4H project which tested the telemedicine approach in 14 regions for patients with heart failure, diabetes and chronic obstructive pulmonary disease (COPD). For diabetic patients in Scotland, they kept in touch with the hospital using text messages. For COPD, some patients used video consultations.

Prof. Crooks stresses that it is not all about the electronics – what matters is the service wraparound that makes the technology acceptable and easy to use for patients and clinical teams.

‘It can take two or three hours out of your day to go along to a 15 minute medical appointment and then to be told to keep taking your medication. What we do is, by using technology, patients monitor their own parameters, such as blood sugar in the case of diabetes, how they are feeling, diet and so on, and then they upload these results,’ said Prof. Crooks.

‘It doesn’t mean you never go to see a doctor, but whereas you might have gone seven or eight times a year, you may go just once or twice.’

Crucially, previous research has shown these patients fare better and the approach is safe.

‘There can be an economic benefit, but really this is about saving capacity. It frees up healthcare professionals to see the more complex cases,’ said Prof. Crooks.

It also empowers patients to take more responsibility for their health and results in fewer unplanned visits to the emergency room.

‘Patient satisfaction rates were well over 90 %,’ said Prof. Crooks.

Humanoid robots under development can be programmed to detect changes in an elderly person’s preferences and habits. Image credit: GrowMeUp

by Helen Massy-Beresford

Low birth rates and higher life expectancies mean that those over 65 years old now will account for 28.7 % of Europe’s population by 2080, according to Eurostat, the EU’s statistics arm.

It means the age-dependency ratio – the proportion of the elderly compared with the number of workers – will almost double from 28.8 % in 2015 to 51 % in 2080, straining healthcare systems and national budgets.

Yet there’s hope marching over the horizon, in the form of robots.

The creators of one humanoid robot under development for the elderly say it can understand people’s actions and learn new behaviours in response, even though it is devoid of arms.

Robots can be programmed to understand an elderly person’s preferences and habits to detect changes in behaviour: for example if a yoga devotee misses a class, it will ask why, while if an elderly person falls it will automatically alert caregivers or emergency services.

Yet there’s still a way to go before these devices will be able to bring out a tray of tea and biscuits when visitors drop by, according to its creator.

At the moment there are things the robot can perform perfectly in the lab but that still present challenges, says Dr Luís Santos from the University of Coimbra in Portugal, who has been developing the technology as part of an EU-funded research project known as GrowMeUp.

The proportion of elderly people is expected to almost double by 2080, so researchers are looking to robots to see if they can help care for the aging population. Image credit: GrowMeUp

‘There is a mismatch between what elderly people want and what science and technology can provide – some of them are expecting robots to do all types of household activities, engage them in everyday gossip or physically interact with them as another human would do,’ says Dr Santos.

The team is working on making the robot’s dialogue as natural and as intuitive as possible and improving its ability to safely navigate an older person’s home, using a low-cost laser and a camera, and a second prototype will be tested with elderly people in the coming months. Yet, Dr Santos foresees that these devices are still four to six years away from commercialisation, at least.

Revolution

He sees robotics as just a part of a wider revolution underway in how societies care for the elderly, with connectivity and augmented reality also playing a role.

‘In the future, elderly care will also be very focused on information and communications technologies – for example virtual access to doctors or care institutions and 24/7 monitoring in a non-invasive way are likely to become standard,’ he said.

Yet researchers believe that keeping the technology unobtrusive is key – no wearable devices or cumbersome cameras cluttering up people’s homes.

Dr Maria Dagioglou from the National Centre of Scientific Research ‘Demokritos’ in Greece, said: ‘We wanted to avoid a Big Brother scenario, so data privacy is important but also dignity.’

She is looking at ways to integrate robotics technology into a smart home equipped with connected devices, automation and sensors, as part of the EU-funded RADIO project.

Researchers are figuring out ways of putting robots in homes for virtual access to healthcare and constant monitoring, yet that are also non-invasive. Image credit: RADIO

Dr Stasinos Konstantopoulos, the scientific manager of the RADIO project, added: ‘All monitoring happens as the user interacts with the system to control the house, for example, to regulate the temperature, and to ask the robot to run errands, like finding misplaced items.’

Using a tablet or smartphone to interact, the equipment, which should only take a day to install, can monitor elements of an elderly person’s day-to-day life, efficiently processing and managing data to allow medical professionals to keep track of and assess their level of independence via smartphone notifications.

‘It’s a constant safety net in case something starts to be worrying,’ said Dr Dagioglou.

The goal of innovations like this is to allow people to live independently for longer.

A crucial element of this is finding ways for older people to keep up their activity levels, and this is an area where robots could really come into their own.

Dr Luigi Palopoli at the University of Trento in Italy said ‘our robot pushes them to do their exercise, to go out and about; it extracts information on their interests and on their fears and makes them part of a network.’

Barriers

‘We want to tear down the emotional barriers that make them stay at home and degrade the quality of their life,’ he said.

As part of the EU-funded Acanto project, he is developing a robot called FriWalk, following on from the progress made during a previous EU-funded project, the DALi project.

The team has worked hard to make the FriWalk robot look energetic and appealing and to ensure it offers useful services like carrying small items or giving directions.

With prototypes made, in the next few months, the researchers will start clinical trials in Spain as well as public demonstrations of the FriWalk in museums and other public spaces.

Researchers will start clinical trials in Spain of a robotic prototype designed to help elderly people to exercise. Image credit: Acanto

Further ahead, Dr Palopoli hopes for interest from established manufacturers and start-ups to bring the FriWalk technology to the market.

If you liked this article, you may also want to read:

•  Choreographing automated cars could save time, money and lives
• 3D-printed houses and cars on the horizon as manufacturing goes large
• Keeping a robotic eye on pollution
• Grow a house with plant-robot hybrids

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Image courtesy of flora robotica, Photo by Anders Ingvartsen, CITA

Robots and plants are being intricately linked into a new type of living technology that its creators believe could be used to grow a house.

‘The growth is for free, but we have to control the plants to grow in the shapes we want,’ explained Professor Heiko Hamann, from the University of Lübeck in Germany, who coordinates the EU-funded flora robotica project. The plants grow through a network of sensors, computers and 3D-printed robotic nodes that are connected to each other and constantly monitor the plants. The team uses a white plastic scaffolding with black strips woven into it to guide the growth. The strips contain LED lights and sensors that can cause plants to grow into pre-programed shapes.

Plastic white scaffolding helps hold up plant-robot hybrids until they are strong enough to support themselves. Image courtesy of flora robotica

Once the plants are strong enough, the scaffolding is removed piece by piece and the hybrids can stand and support themselves. The idea is to create a society of plant-robot hybrids that functions as a self-organising system. As the system grows over long periods in interaction with humans, the research team hopes it will form meaningful architectural forms, including eventually a house.

Prof. Hamann believes a full house will take 40 years to grow, but his team is already starting to create parks where people can design their own plants by mixing seeds, scaffolds and electronics. The techniques required to create the plant-robot hybrids are already well advanced. ‘They can talk to each other … and they already know the pattern that they’re supposed to make, and they can decide which lights should turn on to grow the shape that you actually want,’ he said.

Images courtesy of flora robotica

The robots make use of the fact that plants respond differently to red and blue light. To test this out, researchers put coloured lights around bean plants and monitored their growth.

‘The blue light influences where the plant grows,’ explained Prof. Hamann.

The team is also testing out how to weave different braided structures. Some are loose and let light inside for plants to grow within the structure, while others are woven more tightly making the scaffold stronger for plants and sensors to sit outside.

Image courtesy of flora robotica

The different tightnesses of the braids could also help researchers make bendable arms that are controlled by robots to move side to side and extend and retract – similar to the way a puppet’s arms move when someone pulls a string.

The researchers hope that the arm will be able to interact with people, for example when someone walks by. It’s part of their overall goal to ‘bring technology and nature together’. The idea is that robots will work with people around them, for example by not growing in a place where people often walk by, or telling a passer-by that a plant needs more water.

‘The plant system should interact with human beings,’ explained Prof. Hamann.

Image courtesy of flora robotica

If you take humans out of the driving seat, could traffic jams, accidents and high fuel bills become a thing of the past? As cars become more automated and connected, attention is turning to how to best choreograph the interaction between the tens or hundreds of automated vehicles that will one day share the same segment of Europe’s road network.

It is one of the most keenly studied fields in transport – how to make sure that automated cars get to their destinations safely and efficiently. But the prospect of having a multitude of vehicles taking decisions while interacting on Europe’s roads is leading researchers to design new traffic management systems suitable for an era of connected transport.

The idea is to ensure that traffic flows as smoothly and efficiently as possible, potentially avoiding the jams and delays caused by human behaviour.

‘Travelling distances and time gaps between vehicles are crucial,’ said Professor Markos Papageorgiou, head of the Dynamic Systems & Simulation Laboratory at the Technical University of Crete, Greece. ‘It is also important to consider things such as how vehicles decide which lane to drive in.’

Prof. Papageorgiou’s TRAMAN21 project, funded by the EU’s European Research Council, is studying ways to manage the behaviour of individual vehicles, as well as highway control systems.

For example, the researchers have been looking at how adaptive cruise control (ACC) could improve traffic flows. ACC is a ‘smart’ system that speeds up and slows down a car as necessary to keep up with the one in front. Highway control systems using ACC to adjust time gaps between cars could help to reduce congestion.

‘It may be possible to have a traffic control system that looks at the traffic situation and recommends or even orders ACC cars to adopt a shorter time gap from the car in front,’ Prof. Papageorgiou said.

‘So during a peak period, or if you are near a bottleneck, the system could work out a gap that helps you avoid the congestion and gives higher flow and higher capacity at the time and place where this is needed.’

Variable speed limits

TRAMAN21, which runs to 2018, has been running tests on a highway near Melbourne, Australia, and is currently using variable speed limits to actively intervene in traffic to improve flows.

An active traffic management system of this kind could even help when only relatively few vehicles on the highway have sophisticated automation. But he believes that self-driving vehicle systems must be robust enough to be able to communicate with each other even when there are no overall traffic control systems.

‘Schools of fish and flocks of birds do not have central controls, and the individuals base their movement on the information from their own senses and the behaviour of their neighbours,’ Prof. Papageorgiou said.

‘In theory this could also work in traffic flow, but there is a lot of work to be done if this is to be perfected. Nature has had a long head-start.’

One way of managing traffic flow is platooning – a way to schedule trucks to meet up and drive in convoy on the highway. Magnus Adolfson from Swedish truckmaker Scania AB, who coordinated the EU-funded COMPANION project, says that platooning – which has already been demonstrated on Europe’s roads – can also reduce fuel costs and accidents.

The three-year project tested different combinations of distances between trucks, speeds and unexpected disruptions or stoppages.

Fuel savings

In tests with three-vehicle platoons, researchers achieved fuel savings of 5 %. And by keeping radio contact with each other, the trucks can also reduce the risk of accidents.

‘About 90 percent of road accidents are caused by driver error, and this system, particularly by taking speed out of the driver’s control, can make it safer than driving with an actual driver,’ Adolfson said.

The COMPANION project also showed the benefits of close communication between vehicles to reduce the likelihood of braking too hard and causing traffic jams further back.

‘There is enough evidence to show that using such a system can have a noticeable impact, so it would be good to get it into production as soon as possible,’ Adolfson said. The researchers have extended their collaboration to working with the Swedish authorities on possible implementation.

Rutger Beekelaar, a project manager at Dutch-based research organisation TNO, says that researchers need to demonstrate how automated cars can work safely together in order to increase their popularity.

‘Collaboration is essential to ensure vehicles can work together,’ he said. ‘We believe that in the near future, there will be more and more automation in traffic, in cars and trucks. But automated driving is not widely accepted yet.’

To tackle this, Beekelaar led a group of researchers in the EU-funded i-GAME project, which developed technology that uses wireless communication that contributes to managing and controlling automated vehicles.

They demonstrated these systems in highway conditions in the 2016 Grand Cooperative Driving Challenge in Helmond, in the Netherlands, which put groups of real vehicles through their paces to demonstrate cooperation, how they safely negotiated an intersection crossing, and merged with another column of traffic.

Beekelaar says that their technology is now being used in other European research projects, but that researchers, auto manufacturers, policymakers, and road authorities still need to work together to develop protocols, systems and standardisation, along with extra efforts to address cyber security, ethics and particularly the issue of public acceptance.

Level 3 automation, where the car handles all aspects of driving with the driver on standby, is being tested in Sweden. Image courtesy of Volvo cars

Today’s self-driving car isn’t exactly autonomous – the driver has to be able to take over in a pinch, and therein lies the roadblock researchers are trying to overcome. Automated cars are hurtling towards us at breakneck speed, with all-electric Teslas already running limited autopilot systems on roads worldwide and Google trialling its own autonomous pod cars.

However, before we can reply to emails while being driven to work, we have to have a foolproof way to determine when drivers can safely take control and when it should be left to the car.

‘Even in a limited number of tests, we have found that humans are not always performing as required,’ explained Dr Riender Happee, from Delft University of Technology in the Netherlands, who is coordinating the EU-funded HFAuto project to examine the problem and potential solutions.

‘We are close to concluding that the technology always has to be ready to resolve the situation if the driver doesn’t take back control.’

But in these car-to-human transitions, how can a computer decide whether it should hand back control?

‘Eye tracking can indicate driver state and attention,’ said Dr Happee. ‘We’re still to prove the practical usability, but if the car detects the driver is not in an adequate state, the car can stop in the safety lane instead of giving back control.’

Next level

It’s all a question of the level of automation. According to the scale of US-based standards organisation SAE International, Level 1 automation already exists in the form of automated braking and self-parking.

Level 4 & 5 automation, where you punch in the destination and sit back for a nap, is still on the horizon.

But we’ll soon reach Level 3 automation, where drivers can hand over control in situations like motorway driving and let their attention wander, as long as they can safely intervene when the car asks them to.

HFAuto’s 13 PhD students have been researching this human-machine transition challenge since 2013.

Backed with Marie Skłodowska-Curie action funding, the students have travelled Europe for secondments, to examine carmakers’ latest prototypes, and to carry out simulator and on-road tests of transition takeovers.

Alongside further trials of their transition interface, HFAuto partner Volvo has already started testing 100 highly automated Level 3 cars on Swedish public roads.

Another European research group is approaching the problem with a self-driving system that uses external sensors together with cameras inside the cab to monitor the driver’s attentiveness and actions.

Blink

‘Looking at what’s happening in the scene outside of the cars is nothing without the perspective of what’s happening inside the car,’ explained Dr Oihana Otaegui, head of the Vicomtech-IK4 applied research centre in San Sebastián, Spain.

She coordinates the work as part of the EU-funded VI-DAS project. The idea is to avoid high-risk transitions by monitoring factors like a driver’s gaze, blinking frequency and head pose — and combining this with real-time on-road factors to calculate how much time a driver needs to take the wheel.

Its self-driving system uses external cameras as affordable sensors, collecting data for the underlying artificial intelligence system, which tries to understand road situations like a human would.

VI-DAS is also studying real accidents to discern challenging situations where humans fail and using this to help train the system to detect and avoid such situations.

The group aims to have its first interface prototype working by September, with iterated prototypes appearing at the end of 2018 and 2019.

Dr Otaegui says the system could have potential security sector uses given its focus on creating artificial intelligence perception in any given environment, and hopes it could lead to fully automated driving.

‘It could even go down the path of Levels 4 and 5, depending on how well we can teach our system to react — and it will indeed be improving all the time we are working on this automation.’

The question of transitions is so important because it has an impact on liability – who is responsible in the case of an accident.

It’s clear that Level 2 drivers can be held liable if they cause a fender bender, while carmakers will take the rap once Level 4 is deployed. However, with Level 3 transitions, liability remains a burning question.

HFAuto’s Dr Happee believes the solution lies in specialist insurance options that will emerge.

‘Insurance solutions are expected (to emerge) where a car can be bought with risk insurance covering your own errors, and those which can be blamed on carmakers,’ he said.

Yet it goes further than that. Should a car choose to hit pedestrians in the road, or swerve into the path of an oncoming lorry, killing its occupants?

‘One thing coming out of our discussions is that no one would buy a car which will sacrifice its owner for the lives of others,’ said Dr Happee. ‘So it comes down to making these as safe as possible.’

The five levels of automation:

1. Driver Assistance: the car can either steer or regulate speed on its own.
2. Partial Automation: the vehicle can handle both steering and speed selection on its own in specific controlled situations, such as on a motorway.
3. Conditional Automation: the vehicle can be instructed to handle all aspects of driving, but the driver needs to be on standby to intervene if needed.
4. High Automation: the vehicle can be instructed to handle all aspects of driving, even if the driver is not available to intervene.
5. Level 5 – Full Automation: the vehicle handles all aspects of driving, all the time.

Housebuilders and makers of car parts in a few decades time may need nothing more than a large robotic arm, some raw ingredients and a programmable design, thanks to the next-generation of 3D printing machines which are opening up the technique to large-scale industry.

There are currently two types of manufacturing: additive, also known as 3D printing, where an item is built by adding material layer by layer; and subtractive, which starts with a piece of material and takes pieces away through processes such as milling and polishing until you’re left with the final product.

Now, researchers are building hybrid machines that are able to perform both types of manufacturing at once, and on a scale that brings 3D-printed houses and cars within reach.

‘We want to use the best of both technologies,’ said Jose Antonio Dieste, from Spanish technology centre AITIIP. ‘Additive manufacturing has some problems, one is the accuracy for large parts, another is the quality of the finishing. We are trying to complement this additive with subtractive in order to improve the performance.’

He runs the EU-funded KRAKEN project, which is designing a ceiling-mounted robotic arm that lays down a thin layer of a material, such as aluminium, then finishes that layer by taking away any unnecessary material, before repeating this process until the product is built.

The idea is to accurately construct large components such as side panels for cars, while saving on materials by only using exactly what is required, thus lessening the environmental footprint of industry.

Large components

‘We are going to work on medium and large components,’ he said. ‘[Now] additive machines are working on quite small parts, half a metre maximum. We are going to be able to build 24-metre-long parts.’

New innovations in the nozzles and in the ingredients should also speed up this type of manufacturing, by increasing the deposit rates of the materials, says Dr Dieste.

‘Additive 3D printing machines are very slow. In order to build a half-metre part, some of the machines need one to two days. We need to go faster. For example, a five-metre-long (car) should take two or three days maximum.’

A prototype of the KRAKEN machine, which will be modelled on a robotic arm developed in a previous project called MEGAROB, should be ready by 2019.

It will specifically be designed to build components from metal, particularly aluminium, and resins known as thermosets. However, other researchers are creating hybrid manufacturing machines that can work with other types of materials.

If you liked this article, you may also want to read these other articles on 3D printing:

• Sticky business: Five adhesives tested for 3D printing
• Foundry tool: Multi-material designing for 3-D printing
• 3-D printing hydraulically-powered robots, no assembly required
• AI, deep learning and 3D printing produce ‘The Next Rembrandt’
• Rising Media acquires RoboUniverse and Inside 3D Printing shows

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Image courtesy of FIXO3

The shells on crustaceans and molluscs off the Norwegian Atlantic coast are not as thick as they once were. This is because of ocean acidification, where increased carbon dioxide (CO2) in the atmosphere is absorbed by water, which raises its acidity. Researchers on the EU-funded FixO3 project, which finishes in 2017, are able to keep track of the growing CO2 and rising acidity levels in the water where these creatures live thanks to a device in the Norwegian Sea which collects data round the clock from as deep as 2 000 metres.

Image courtesy of EMSO

Similar tools are being set up all around Europe as part of the European Multidisciplinary Seafloor and water-column Observatory (EMSO), which is part of the EU’s centrally coordinated research infrastructures, to measure how human activities affect our oceans and worsen climate change. As well as providing food and producing oxygen through the organisms that live within them, our oceans and seas regulate our climate by transporting warm water from the equator to the poles and cold water in the other direction. Researchers are using sophisticated machines to track how our oceans are changing, from small cameras that can record high-definition videos of the ocean floor, to others that can detect earthquakes, measure temperature and pressure, and record sounds.

Image courtesy of EMSO

Data is transmitted from machines at the bottom of the ocean either through fibre optic cables or via buoys on the surface, which are linked to satellites. Scientists on land can then track this data, either in real time or with a delay depending on the system, and monitor pollution, climate change and even tsunamis over time. While the machines do require regular maintenance, it means that researchers are able to study the long-term health of our oceans while cutting down on costly expeditions.

Image courtesy of EMSO

Some of the findings to come out of these observatories are expected to be included in future reports by the UN’s influential Intergovernmental Panel on Climate Change. EMSO’s tools, such as one monitoring seismic activity and pressure on hydrothermal vents in the Mid-Atlantic Ridge, will also help the EU know if it has reached its goals of improving the health of Europe’s seas by 2020, as set out in the EU’s Marine Directive. EMSO is also the European counterpart of initiatives in the US, Japan, China, Australia and other countries and will help with international collaboration in the Global Ocean Observing System, a worldwide effort to track changes in our waters.

Image courtesy of EMSO

Similar tools are being set up all around Europe as part of the European Multidisciplinary Seafloor and water-column Observatory (EMSO), which is part of the EU’s centrally coordinated research infrastructures, to measure how human activities affect our oceans and worsen climate change. As well as providing food and producing oxygen through the organisms that live within them, our oceans and seas regulate our climate by transporting warm water from the equator to the poles and cold water in the other direction. Researchers are using sophisticated machines to track how our oceans are changing, from small cameras that can record high-definition videos of the ocean floor, to others that can detect earthquakes, measure temperature and pressure, and record sounds.

This article was first published on Horizon: The EU research and innovation magazine. Click here to view the original article.

If you liked this article, you may also be interested in:

• Unmanned and underneath: Stay in the know about underwater drones
• What you need to know about underwater drones
• Autonomous underwater system for monitoring health of Venice Lagoon demos at #EXPO2015
• CoCoRo: Tracking the development of the world’s largest autonomous underwater swarm
• Ocean-going robot fleet successfully completes fish-tracking mission

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Helium balloons are being used to carry people, experiments and satellite launchers into near space. Image courtesy of Zero2Infinity

Space will soon be within the grasp of everyday people, small countries, researchers or start-up companies thanks to a fleet of low-cost launch vehicles under development across Europe.

From remote control drones that can fire rockets into orbit from the edge of space, to giant helium balloons that could carry tourists, science experiments or satellite launchers into the stratosphere, small-scale start-ups are stepping into the control room and embarking on a new low-cost space race.

It represents an exciting new wave of space travel, following in the vapour trails of private companies such as Elon Musk’s SpaceX or Richard Branson’s Virgin Galactic.

The Issue

The EU is aiming to encourage private companies to develop low-cost launch systems as part of its Space Strategy for Europe announced in October this year.

The objective of EU funding for research into space technologies is to enable Europe to access space independently and affordably, and strengthen the capability of European scientists and engineers.

Between 2014 and 2020, it plans to spend over EUR 12 billion on three space programmes – satellite navigation, earth observation and space research.

Where once only global superpowers with vast budgets could go, Europe’s entrepreneurs and innovators will soon be travelling for a fraction of the cost.

One of these innovators is Zero 2 Infinity, a Spanish start-up working on helium balloon-borne spacecraft.

‘It’s the cheaper way because it’s more efficient, you need fewer engineers to build it,’ explained Jose Mariano Lopez-Urdiales, chief executive.

Near space

The idea is that the balloon would carry a passenger pod into what is known as near space, somewhere between 20 km and 100 km above the earth’s surface, or carry its small rocket-powered Bloostar spacecraft which it could then launch much higher into low-earth orbit.

Zero 2 Infinity customers are already balloon launching satellites for short periods of high-altitude testing, and the company has EUR 140 million in expressions of interest for full satellite launches.

It was the leader of the EU-backed HELIUM project, which developed the concept for a near-space European laboratory that allows orbiting research at 36 km, and the life support system for the two pilots in the initial six-hour flights.

Although space tourists could soon be using the balloon launch system to take selfies in near space, the first passengers are likely to be scientists, according to Lopez-Urdiales.

Aiming higher than the balloon satellite launcher’s range of 600 km, Europe’s ALTAIR project is readying the blueprints for a remote-controlled aircraft which would use a rocket launch module to blast sub-200 kilogramme satellite payloads into orbit and take off in 2020 at the earliest.

Runway

ALTAIR’s launch system — similar to Virgin Galactic’s piloted White Knight 2 space tourism aircraft in appearance and general function — would take off from a runway, before ferrying satellites up to an altitude of 12-15 km and igniting its rocket payload.

But the similarities end there. ALTAIR project manager Nicolas Bérend says the project is focused on affordable, remote-controlled launches, calling on the experience of French space centres CNES and Onera, creators of the Eole automated launch aircraft demonstrator which is the project’s precursor.

By the end of the EU-funded program in 2018, the six-country collaboration will have a final concept for the unmanned aircraft, its hybrid propulsion engine rocket launcher and ground operations, and will demonstrate its launch technology by doing suborbital tests using a scale model.

By removing the human pilot and designing a facility geared towards a high launch frequency and minimal staffing, they hope to bring down the costs enough to make the project economically viable.

‘It is essential to the project’s success to restrict launch costs to around EUR 4 million, a price the team’s market analysts consider to be a competitive launch price,’ said Bérend.

That should give them access to a growing market for small satellite launches, often in the form of so-called CubeSats, nanosatellites normally no more than 10 centimetres wide, a market where Europe is lagging behind at the moment.

Even though Europe covers large-to-medium launches with its Ariane 6 and Vega rockets, and suborbital rockets from the Andøya launch site in Norway, it has no small-satellite solution.

‘The small satellites are all launched outside of Europe with Russian or Indian launchers,’ said Bertil Oving, of the Netherlands Aerospace Centre. ‘There is a small-satellite market, it’s booming and if Europe does not take the chance now, then we may lose this possibility.’

He is the coordinator of SMILE, an EU-funded project designing a small-scale launcher for satellites of up to 50 kg.

SMILE will design and start component testing of a standard-format rocket, along with the necessary upgrades to the Andøya facility, by 2019, keeping costs below EUR 50 000 per kilogramme launched, and hopefully cutting the expensive wait for satellites to be able to piggyback on one of the larger rockets.

3D printing

Larger rockets benefit from an economy of scale, and SMILE’s competitiveness would come from advances in reusable components, automated composite manufacturing, 3D printing and also high-volume production.

‘The payload versus the total mass becomes smaller when the launcher is smaller, so it’s a cost aspect challenge,’ said Oving.

He said SMILE is pushing advances in reusable liquid-fuelled engines and its automated manufacturing advances would have automotive and aviation industry uses, but he thinks the biggest implications of low-cost, low-orbit access will be for consumers.

‘The idea for small satellites like CubeSats is to approach them like FedEx & DHL,’ he said. ‘We will simply bring it to where you want to have it.’

European organisations may soon make it easier to get equipment into near space and low-earth orbit.

If you enjoyed this article, you may also want to read these Horizon related articles:

• EU’s Horizon 2020 has funded $179 million in robotics PPPs
• Keeping a robotic eye on pollution
• Protecting European wine: Vinbot rover optimises harvest and quality
• A helping hand for high-tech firms

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

By Ethan Bilby.

Field robots and plane-based remote sensors can patrol the earth and the sky to monitor the gases that cause climate change. Standing on three large wheels that help it avoid getting stuck in the soil, the Field Flux robot is able to lower two sampling chambers held on large arms to test soils for tiny amounts of nitrous oxide (N2O).
Though carbon dioxide (CO2) is better known when it comes to climate change, N2O has a global warming potential that’s over 300 times higher. In other words, a single molecule of N2O is as damaging as 300 CO2 molecules.

‘One of the great difficulties in quantifying N2O emissions from agriculture is that it changes with a factor of 100 from one day to the next and one spot to another,’ said Professor Lars Bakken, from the Norwegian University of Life Sciences in Oslo. He is a microbial ecologist on the NORA project, which is working out ways to reduce N2O emissions from soils and developed the monitoring robot alongside Norwegian company Adigo. ‘This is actually the whole reason we made the field robot. If you want to quantify emissions from a field experiment you have to measure over and over and over in many plots,’ said Prof. Bakken.

The Field Flux prototype can measure the amount of the greenhouse gas nitrous oxide in agricultural fields. Image: NORA

Using the robot, a field that took 27 hours to test manually can be done in just one hour. Those measurements are crucial in controlling N2O as they could allow farmers to treat their soils where needed. The N2O is produced by some soil microorganisms during periods when the soil is not well aerated, for example during heavy rain, or when the soil is tightly packed. Those organisms, mostly bacteria, then use nitrogen oxides instead of oxygen for respiration. Several of those bacteria can respire N2O by using a very special enzyme, the N2O reductase. The researchers in the NORA project discovered that this enzyme is prevented from functioning correctly in acidic soils, and also when there isn’t enough copper present.

‘We looked at the biology of these organisms and how they regulate those processes that are responsible for N2O emissions,’ said Professor Åsa Frostegård, also a microbial ecologist at Norwegian University of Life Sciences, and the coordinator of the Marie Skłodowska-Curie Actions-funded project. ‘We see large differences in how different microbes do these things.’

Image: ADIGO

Armed with that knowledge, it could be possible for farmers to reduce N2O pollution by changing the acidity or copper content in their soils. That could mean using so-called mafic rocks, minerals that are rich in iron and magnesium, to neutralise acid in the soil, instead of traditional liming, which can increase CO2.

Gazing at clouds

Monitoring particles once they get into the atmosphere, where they are known as aerosols, isn’t any easier. Researchers in the Marie Skłodowska-Curie Actions-funded ITaRS project, led out of the University of Cologne in Germany, are using remote aircraft-based sensors and ground-based measurements to get a handle on when clouds are likely to form or develop precipitation.

‘One of the major uncertainties in atmospheric models is how clouds interact with aerosols in the atmosphere. We don’t even know in detail how clouds are formed,’ said Dr Maria Barrera Verdejo, an ITaRs fellow working with lidars and microwave radiometers. To form, clouds need particles such as dust or pollen for water to condense around. Improving the precision of ground- and air-based measurements provides researchers with new data they can use to better understand atmospheric phenomena, for example how and under which conditions storms will develop. ‘My measurements can be used for something that is called data assimilation in weather forecasting. You feed in what the atmospheric state is, and the model calculates what is going to happen next,’ Dr Barrera said.

With the measurements deployed within the ITaRS project, researchers were able to answer some scientific questions related to the understanding of atmospheric events. For example, information about what humidity, pressure conditions and particle concentration are required for a cloud to start drizzling, was obtained thanks to passive microwave sensors combined with radar measurements.

That data provided by ITaRS not only improves our knowledge on the atmospheric behaviour, but also helps to reduce uncertainties on climate models, which translates to a better understanding of climate change.

This article was originally published by Horizon Magazine.

If you enjoyed this article, you may also want to read:

• Swarms of precision agriculture robots could help put food on the table
• Top 10 technologies in precision agriculture
• Farming with robots
• Protecting European wine: Vinbot rover optimises harvest and quality
• Robots for a sustainable future

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

L’Aquila, Abruzzo, Italy. A government’s office disrupted by the 2009 earthquake. Source: Wikipedia Commons

by Fintan Burke

‘There’s no real way to determine how safe it is to approach a building, and what is the safest route to do that,’ said Dr Angelos Amditis of the Institute of Communication and Computer Systems, Greece. ‘Now for the first time you can do that in a much more structured way.’

Dr Amditis coordinates the EU-funded RECONASS project, which is using drones and satellite technology to help emergency workers in post-disaster scenarios.

It’s part of a new arsenal of airborne disaster response technologies which is giving a bird’s eye view of disaster zones to help save lives.

The system developed by RECONASS places wireless positioning tags into a building’s structure, along with strain, temperature and acceleration sensors, either when it is first built or through retrofitting. Then after a disaster strikes drones are deployed to scan the building’s exterior and match data from the sensors to this image to build an accurate representation of the damage.

This allows the system to precisely calculate potential areas of structural weakness in the damaged building, and it can then be paired with satellite data to get an overview of the damage in an area.

‘From one side we have 3D virtual images showing the status of the building … produced by ground sensors’ input, and from the other side we have the 3D real images from the drones and satellite views,’ said Dr Amditis.

‘There’s no real way to determine how safe it is to approach a building, and what is the safest route to do that.’

Dr Angelos Amditis, Institute of Communication and Computer Systems, Greece

Dr Amditis and colleagues are hoping to test the system in September by detonating a mock three-storey building in Sweden to investigate how well the drone technology works.

The end goal is for emergency services to use the drone technology to provide information about the structural state of tagged buildings within the first crucial hours after the disaster.

To boost rescue services’ ability to respond quickly in the event of a crisis, researchers in Spain are working on how to help emergency helicopter pilots successfully navigate through a disaster area by giving them more precise and reliable information about their position.

The EU-funded 5 LIVES project is building a system that uses both the newly launched Galileo Global Navigation Satellite System (GNSS) from the European Commission and the European Geostationary Navigation Overlay Service (EGNOS), which reports on the reliability and accuracy of data from satellites.

At the moment, most European helicopters use visual navigation, meaning they cannot fly in bad conditions such as fog, clouds or bad weather.

While more and more are beginning to use GNSS technology, the system has been known to be affected by slight changes in the earth’s atmosphere. There is also no way of knowing how accurate the information is.

‘With conventional GNSS technology positioning, there are a lot of external effects that might negatively affect the precision of positioning,’ said project manager Santi Vilardaga of Barcelona-based Pildo Labs.

In order to encourage the adoption of GNSS technology, the 5 LIVES system overlays information from EGNOS to confirm the precision and accuracy of satellite data.

‘Different parameters that affect the displacement of the signal or the propagation of the satellite signal through the atmosphere are collected through a network of ground-based stations and then transferred to the geostationary satellites that then broadcast the corrections to the EGNOS users,’ he said.

More flexible

This extra precision and confidence in the data allows emergency helicopters to become more flexible. ‘They can operate at night, they can operate technology in meteorological conditions without need of added technology on the ground,’ said Vilardaga.

They are also designing drone-based search and rescue operations. Drones can be used to fly around disaster areas, using on-board cameras to continue the search for survivors even when weather conditions make it unsafe for human pilots to do so, explains Vilardaga.

The team is now ensuring that the drones satisfy existing European regulation and their new positioning technology is up to standard for a demonstration of their search and rescue procedures next year.

The need for a new way to combat natural disasters from the air is being driven in part by a new type of crisis – so-called mega-fires. These once-rare fires, which are known to spread at a rate of more than three acres per second, are forest fires large enough to spread between fire-fighting jurisdictions.

A variety of factors such as drought, climate change and the practice of preventing smaller fires to develop naturally have all been blamed for the increasing frequency of mega-fires.

To defend against these fires, the EU-funded Advanced Forest Fire Fighting project is trying to improve both firefighting logistics and technology.

One method involves developing a pellet made of extinguishing materials which can be dropped from a greater height, meaning pilots are less likely to encounter risks from smoke or heat from the fire.

Another area under development is centralising the information from satellite data, drones and ground reports into what is being called a core expert engine to help firefighters to prioritise procedures.

‘The commander can have at his disposal just one aircraft with a pellet,’ said project coordinator Cecilia Coveri of Finmeccanica SpA, Italy, ‘and in case the pellet is the best way to extinguish a fire … this simulation can help the commander to take this decision.’

The project begins its first trials in Athens during the summer in cooperation with Greek naval forces, with additional testing of the risk analysis and simulation tools to follow.

If you enjoyed that article, you may also enjoy reading: 

• Robot bees open the lid on hive behaviour
  
• Protecting European wine: Vinbot rover optimises harvest and quality
• Fish and ships: artificial lily pads and mussels monitor Europe’s waters
• Nature inspires new generation of robot brains
• Octopus points to the future for keyhole surgery

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Researchers are studying bees to make robotic versions that can help beekeepers look after their hives. Image courtesy of ASSISIbf

by Steve Gillman

It’s part of a series of projects where researchers are building robots that mimic evolutionary traits of animals in order to interact with the world in a more efficient way.

‘What we had in a mind was a set of machines that were capable of interacting with bees,’ said Dr Thomas Schmickl of the Artificial Life Laboratory in Graz, Austria. He coordinates ASSISIbf, an EU-funded project to design robots that integrate with animal societies for environmental and agricultural monitoring.

‘We want our robots to be accepted by bees so they are treated like real members of the society.’

The project has a bio-hybrid hive where static robot bees exist side-by-side with their living counterparts, so far for periods of around two to three hours. The idea is to paint a picture of what is going on inside the hive and then use the robots to influence the bees’ behaviour.

‘You soon notice that a dialogue emerges,’ said Dr Schmickl.

The robots can artificially recreate several stimuli such as airflow from fanning wings and temperature, which can be both appealing and repelling for the living bees. They can also vibrate and mimic the queen bee, which speeds or stops the living organisms.

Sensors on the living bees also feed into an ‘evolutionary algorithm’ which learns how to map their behaviour and allows the robotic bees to better interact with the insects.

‘With just a few (robot) bees we can detect abnormalities, something that was different the days before,’ said Dr Schmickl. ‘You can retrieve information about the whole situation of the population: is it dense, is it moving or not? Then you have an environmental sensing capability.’

This can help beekeepers look after their hives better, in particular, reducing the need to open them for inspection which can cause stress to all the bees and even kill hundreds of them each time.

‘We want our robots to be accepted by bees so they are treated like real members of the society.’

Dr Thomas Schmickl, Artificial Life Laboratory in Graz, Austria

‘In the beginning it’s just a beehive, but in the long run, the more sophisticated this gets, you could be talking to the whole population of a forest,’ said Dr Schmickl. ‘We want to get control over the animal community by establishing a fair dialogue between the animals and the machines.’

Agriculture is not the only industry that could benefit from bio-inspired robots: ocean energy could also use the evolutionary traits of squid and octopus.

‘Ocean renewable energy technologies, such as offshore wind farms or wave energy harvesting devices, are designed to produce energy from stretches of sea where winds are stronger and waves are higher, but while extreme weather conditions increase the productivity it also increases the costs of maintenance of these structures,’ said Dr Francesco Giorgio-Serchi from the University of Southampton, UK.

Currently, maintaining these huge structures requires taking them out of the ocean, but a lot of time and money would be saved if you could access them underwater.

‘Squids and octopus are able to swim in a very efficient way,’ said Dr Giorgio-Serchi. ‘They can perform very impressive manoeuvres without having joints … so we tried to do the same thing.’

Dr Giorgio-Serchi ran the EU-funded CFD-Octoprop project, which designed an octopus-inspired robot with a rubber cavity instead of organic tissue and cables instead of muscles, which were pulled with a small electric motor.

‘By shortening the cables, the shell would shrink and expel the fluid inside, then simply because of the elastic nature of the rubber material, the shell would re-inflate, suck water in and the cycle would start again,’ he said.

This recreated an octopus’ propulsion which gives it a manoeuvrability that could see it act as a future deep sea repair robot, and that’s not all.

‘They could be used for disaster response and to check the wellbeing of the flooded parts of nuclear power plants,’ said Dr Giorgio-Serchi.

‘The shape-changing effect, which plays such a big part in the generation of thrust of these vehicles, could also have an impact on the design of new energy-generating devices for underwater energy harvesters.’

1 / 4
Dr Serchi and his team made a robot prototype that looks like an octopus with the hope that a final version can act as a deep sea repairman. Image credit: Age of Robots, M. Brega

2 / 4
The researchers are inspired by the octopus because this creature can easily move and manoeuvre in the water without any joints. Image credit: Age of Robots, M. Brega

3 / 4
The researchers put cables and a small motor inside a flexible rubber shell, which can easily fill up and expel water. Image credit: Age of Robots, M. Brega

4/ 4
The robot pulses through the water leaving a wake behind. Image credit: CFD OctoProp

Unforeseen applications

Mimicking the movement of an animal can open the door to a range of new technologies with previously unforeseen applications, say researchers.

‘(Bio-inspired) robots could help us build artificial limbs, exoskeletons or robotic companions,’ said Dr Helmut Hauser from the University of Bristol, UK and previously of the University of Zurich in Switzerland. While in Zurich, Dr Hauser was project manager of Locomorph, an EU-funded project which developed animal-inspired movement in robots.

‘A lot of the control and processes are not happening in the brain, but in the body itself. The body is carrying out computational processes.’

Locomorph constructed over 20 bio-inspired robots aiming to increase self-stabilisation, energy efficiency and manoeuvrability in unknown environments.

‘We looked at ecological methodologies that were caused by natural selection and built different terrains in the lab, like slopes and stairs,’ said Dr Hauser. ‘Also a very important concept of the project was the idea of morphosis, i.e. changing the body to adapt to new tasks and different environment.’

For example, Locomorph looked at dogs, toddlers and apes as well as different types of salamanders, some that lived in the grass and others that lived in water.

‘They have very basic principles that can be found in humans, but we are much more complex. If we understand salamanders we can build upon those principles,’ said Dr Hauser.

The idea is to build robots whose movements are suited to different types of situations, such as search and rescue operations or working on a factory floor.

Researchers are creating robots that can move in a variety of environments, just like the animals that inspire them. Images courtesy of Locomorph

Locomorph developed a whole range of new control mechanics and robotic systems that implemented these underlying principles. Their results are available in a robot construction kit, called LocoKit, which allows other roboticists and researchers to build robots with legs and locomotive ability.

‘As these systems become more intelligent, sooner or later, I think we will have robotic companions working with us at home,’ said Dr Hauser.

See also

• Octopus points to the future for keyhole surgery
• Inspired by nature – made by science
• Nature inspires new generation of robot brains
• Fish and ships: artificial lily pads and mussels monitor Europe’s waters

Artificial lily pads store solar energy and collect information such as shipping routes for robotic fish and mussels below the water’s surface. Image courtesy: subCULTron

by Fintan Burke

Europe’s waters have never been busier. Between fishing boats, tourist cruisers, and underwater turbines, marine habitats around the continent have never before needed a more watchful eye.

To improve the monitoring of these waters, researchers are recreating the ecology and unique senses of marine life.

The subCULTron project is creating a network of autonomous robots which occupy the roles of a lily pad, a fish, and a mussel respectively.

The project uses ‘swarm’ technology, where a group of robots communicate with each other to build up a comprehensive picture of the environment without relying on a single source of information.

‘If you lose 20 % of your swarm robots, it means 80 % of your swarm is still collecting data and is still doing some work,’ said project leader Dr Thomas Schmickl of the University of Graz, Austria. ‘If you lose your one field robot, then game over.’

On the surface, autonomous lily pad robots, known as aPads, use their large area to host solar power cells on their surface. ‘This is like an energy recharging unit we have,’ said Dr Schmickl.

As well as providing energy, the aPads also receive information from the outside world, such as shipping routes. This allows the other robots to be fully aware of their environment.

Beneath the surface, aFish robots are designed to swim like regular fish to monitor water quality and marine life. ‘They are information carriers and they are also seeking for new information, and when they meet each other they can exchange information,’ said Dr Schmickl.

The aFish also have electrodes on their surface which project an electric field around themselves. Any particle entering this field will distort it, which is ‘sensed’ by the robot to provide information about the surrounding area.

Lastly, aMussels on the ocean floor serve as the long-term memory of the ecosystem, storing data as well as sampling some sediment.

‘They can be woken up by the aFish as soon as they get close by and then the aFish can store information in this mussel, can retrieve information from this mussel and use this mussel as a sort of communication platform,’ said Dr Schmickl.

Researchers made a robotic mussel to collect information on the ocean’s floor. Image courtesy: subCULTron

One emphasis of the project was to ensure that the system survives on its own for as long as possible. Dr Schmickl hopes the robots will achieve a high level of interaction to keep the system dynamic and self-sustaining.

‘The aMussel will surface so it will go up to the surface, call for an aPad, and the aPad will go there and charge this aMussel and then maybe also displace the aMussel to a more favoured place,’ he said.

The robotic mussel floats up and docks in the artifical lily pad to charge. Image courtesy: subCULTron

Following on from successful lab trials in a previous project, the team is conducting field trials in Venice, Italy, to see if the robots can survive in a natural environment.

‘This is a really out-of-the-lab condition and to our knowledge a (fully autonomous) swarm robotic application has never been released to the wild,’ said Dr Schmickl.

Lateral line

Another project, LAkHsMI, uses another unique type of bio-inspired technology to monitor the ocean. Marine vertebrates have what is called a lateral line – sense organs that detect movements and vibrations in the surrounding water.

According to Professor Maarja Kruusmaa of the Tallinn University of Technology, Estonia, many researchers have tried to replicate this bio-inspired technology before, though nobody has attempted it at the scale her team is developing.

‘We want to build a gigantic artificial lateral line, something that could sense flow on an area, for example, a 10-kilometre-long line or at least a 100-metre-long line,’ she said.

The sensor uses fibre optic technology throughout the cable in order to detect changes in velocities deep underwater. When the cable is bent due to underwater forces, the light running throughout the cable changes intensity. Researchers measure this intensity change to detect what is going on under the water.

‘The advantage of this technology is that it doesn’t need to have a power source at the spot of sensing, because it just transmits light from the shore and you receive the light from the shore,’ explained Prof. Kruusmaa.

Using these cables to measure the lower levels of the ocean will also provide new information to oceanographers.

‘If you could put a cable across the strait you could understand how the bottom layers of water are flowing,’ she said. ‘Now if you match it with a satellite imagery you get a more complex understanding of the whole circulation in the ocean.’

For now, the LAkHsMI team are continuing to develop their prototype at the lab scale, with the later aim of testing the sensor at sites in both Estonia and Orkney, UK.

One potential application, says Prof. Kruusmaa, is checking how much of the ocean’s energy can be captured by underwater turbines. ‘One thing with renewables, what you really want to know, is how much energy of the ocean does it use and how much (the turbine) disturbs (the) environment’ she said.

Systems based on animal neurons could help robots understand the world. Image credit: CC0

by Rex Merrifield

Animals have evolved sophisticated ways of processing sensory data to make sense of their surroundings. Now, robotics researchers are drawing inspiration from biological processes to improve the way machines handle information, perceive their surroundings, and react to stimuli.

While the human brain is often seen as the ultimate model for robotic intelligence, scientists are also learning plenty from the neurobiological structures and processes of more humble creatures, from fruit flies to rodents.

Take the fruit fly – or rather, the maggot that grows up to be a fruit fly. Drosophila fruit fly larvae have fewer than 10 000 neurons – compared to about 100 billion in the human brain. But they display a range of complex orientation and learning behaviours that computational theory does not adequately explain at present.

By studying how the larvae change their response to stimuli such as smells when these are associated with reward or punishment, the EU-funded MINIMAL project aims to unpick the exact mechanism underlying learning processes.

‘We think it is quite important in any biological system to look at how the way the animal is acting and what it does has an effect on what it senses and what it learns, and how that influences its behaviour the next time,’ said Professor Barbara Webb, professor of biorobotics at the University of Edinburgh, UK, and coordinator of MINIMAL.

‘Animals have to learn while they survive – they can’t separate those two things. For many robot applications it may be equally important that they can learn rapidly while on the job,’ she added.

Using genetic methods that allow specific Drosophila neurons to be tagged, the researchers are able to watch and manipulate how these processes work in real time, identifying the critical neural circuits and providing insight into a quite fundamental way of learning, which they are then applying to robotics.

Being able to process sensory data in real time and react quickly is crucial to animals’ survival, and the REAL-TIME ASoC project is drawing on biological processes for algorithms that make robots more efficient in perception and in search functions.

Project researcher Dr Francisco Barranco, of the University of Granada in Spain, is working with electronic sensors that only transmit information from pixels that detect changes, a far more efficient process than transmitting the full picture continuously.

Human eye

The robot vision processor provides a kind of ‘bottom-up’ filter along the lines of the human eye – where information from 100 million sensors in the retina needs to be conveyed to the brain through the optic nerve, which has only about 1 million ‘cables’ to carry the information. Efficient filtering at the retina is common even in animals such as insects.

The system could have many different applications, such as in interactive television, but there is a long way to go to get the technology from the robotics laboratory to a marketable product.

‘We have developed a low-level vision engine, for biologically inspired vision sensors.’

Prof. Eduardo Ros, University of Granada, Spain

‘We have developed a low-level vision engine, for biologically inspired vision sensors,’ said Professor Eduardo Ros, also at the University of Granada, who supervised the EU-funded research. He said the research is also looking at more sophisticated stages of the vision process typically found in mammals, such as motion processing and depth estimation, and attention.

‘We want a robot to be able to select the important information,’ he said. ‘It needs to be able to discard all the other irrelevant sensations early on, otherwise it becomes overwhelmed.’

Prof. Ros says that visual perception in mammals is as much about a part of the brain called the cortex as it is about information coming through the eye, a feature they are trying to replicate in robots.

Searching for objects

When searching for an object, such as a moving red car, the robot needs to have a clear idea of what it must find, to limit the possibilities.

So REAL-TIME ASoC gives the robot prior information, such as the shape, size, colour or texture that it can work from. It also has to be able to identify how an object may be moving around. By combining these two modes of processing – bottom-up and top-down – the robot is able to find the target objects.

Crucial to successful physical interaction between robots and objects is accurate spatial perception, and the EU-funded GRIDMAP project is drawing on studies of brain anatomy, animal behaviour, and mathematical modelling to engineer a robotic system that can perceive, memorise, and react to changes in its environment in a similar way to rats.

The researchers are studying rats to make computer models of grid cells and place cells – the neurons in specific regions of the mammal brain that help us to perceive and retain an idea of the space in which we live and move. In 2014, GRIDMAP’s coordinator, Professor Edvard I. Moser, based at the Norwegian University of Science and Technology in Trondheim, was awarded the Nobel Prize in Physiology or Medicine jointly with two other neuroscientists for work they had done previously in identifying this ‘inner GPS’ system in rats.

The GRIDMAP researchers now want to understand how these cells form and build up spatial representation, starting from a single cell to regular, observable patterns of activity, said Professor Jörg Conradt, professor of neuroengineering at the Technical University in Munich, Germany, who helps in coordinating the project.

While the group in Trondheim is observing how the rat brain’s spatial representation changes, readjusts and adapts as its immediate environment is made more and more complicated, Prof. Conradt and his colleagues in Munich have built robots that learn to navigate like rats.

‘In the robots experiment, they explore the arena, they observe and they build a representation, so the software running on our computers builds spatial representations just as we believe biology builds up spatial representations over time,’ Prof. Conradt said.

By using cameras and computer controls, the robots learn to find their way through changing mazes as these are made more and more complicated.

‘So it is a rat’s world, adapted for the robot’s senses,’ Prof. Conradt said.

Understanding the fundamental principles in grid cells and place cells could help to construct new systems that are able to achieve self-creation of software – or machines that can programme themselves.

Keyhole, or minimally invasive, surgery can offer many benefits over more traditional, open operations, including reduced risk of infections, quicker recovery times and less scarring. But internal organs can get in the way when hard robotic arms are used, given that access can be very limited and soft tissue can sometimes move in unexpected ways.

The EU-funded STIFF-FLOP project has designed, built and operated a soft robotic arm that can squeeze into the body, manoeuvre gently around soft tissue, reconfigure itself, and stiffen to perform tasks that need force.

‘The aim is to develop soft robotics systems that can elongate, bend and move around organs,’ said project coordinator Professor Kaspar Althoefer who conducted the research at King’s College London, UK, and is now at Queen Mary University of London, UK.

‘The octopus provided some inspiration. It has no bones or skeletal structure, and it can squeeze through very narrow openings, but stiffen when required,’ he said.

The robotic arm works through a combination of air and granules. When air is pumped in, the granules inside it are able to move freely and allow both flexibility and a large range of motion. When the air is sucked out again, the granules clump together, making the structure stiff and rigid in the required position.

The system also helps surgeons to deal with complex soft tissue movements and uncertain situations, such as when organs shift, or move with the patient’s breathing and heartbeat.

With keyhole surgery, the instruments are inserted through narrow openings and they have to pivot around that small incision. And because the surgeon is unable to use a hand directly to feel inside the body, for example to identify the borders of a tumour, a successful operation is dependent on visual feedback from a camera at the end of the robotic arm.

Researchers have developed a prototype of an arm that can stiffen or become flexible on demand. Image courtesy of StiffFlop

So among the next steps is developing technology allowing surgeons to receive feedback from the operation site through touch, a field known as haptics.

In addition to joysticks and data gloves, haptic devices include possibilities such as vibratory actuators and pressure pads on the surgeon’s arm to simulate the feeling of operating through a much larger incision.

Dr Helge Wurdemann, project manager of STIFF-FLOP, said: ‘In particular, haptics has great potential to offer benefits to surgeons but, more importantly, to patients’ outcomes, when it comes to interventions that are less and less invasive.’

Cadaver

The STIFF-FLOP project put the soft robotic arm through a series of successful tests on a cadaver. The collaborators are now developing it further, with a view to possibly testing it in veterinary surgery, and eventually in operations on live people.

‘We have demonstrated the feasibility of the approach and so we would like to try it out on a living system, though that will require much further development,’ Prof. Althoefer said.

Other EU-supported researchers have been working to improve the software that controls surgical robots and tools such as graspers or scalpels and provide the surgeons who use them with the sense of touch when tissue is palpated, grasped or cut.

‘Any kind of design choice, any kind of device, any new control system, has to be grounded in what we as motor neuroscientists understand about how people control motion and perceive information,’ said Dr Ilana Nisky, senior lecturer in the Department of Biomedical Engineering and head of the Biomedical Robotics Lab at Ben-Gurion University of the Negev, Israel.

It can squeeze through very narrow openings, but stiffen when required. Professor Kaspar Althoefer, King’s College London, UK

‘From the side of the surgeon, we are developing the software that will make them feel as if they are present there, so that it is really natural,’ added Dr Nisky, whose TELEPRESENCE SURGERY project has been developing new control software for existing robotic surgical equipment.

Her research has found that very experienced surgeons operating with robotic equipment use a far greater ‘toolbox’ of hand and arm movements in carrying out operations than novices do. While experience obviously brings increased skill, the contrast was much bigger than expected and was present even in the performance of very basic motor tasks that do not require surgical knowledge. Those findings provided inspiration for further developments in robotic arms and controls.

‘We hope that eventually robots will make it easier to learn how to use them too,’ Dr Nisky said. ‘Lead surgeons can do marvellous things with the robots, and by improving the controls and systems we hope that will shorten the learning curve for novices and they will be able to master complicated procedures faster.’

More info:

STIFF-FLOP

TELEPRESENCE SURGERY

Drone flying over a field.

by Fintan Burke

The last place you might expect to find drones and rovers is checking up on a corn field, but they could soon join tractors and ploughs on a farmer’s list of must-have agricultural tools, thanks to their potential to reduce pesticide use and increase the amount of crops that can be grown.

It’s part of a move towards so-called precision farming, where farmers use equipment to monitor their crops and respond to issues when and where they occur, rather than applying blanket solutions to an entire field.

The Issue
The dangers of excessive pesticide use on the environment can be traced back to the 1962 book ‘Silent Spring’ by Rachel Carson, which has been credited as starting the modern conservation movement.
The EU now applies restrictions to certain pesticides containing neonicotinoids, which have been linked with the collapse of honey bee populations throughout Europe.
In 2014, an EU study highlighted the benefits of precision farming. Using this method can reduce pesticide and fertiliser runoff, which can greatly damage the local environment

Dr José M. Peña of the Institute for Sustainable Agriculture in Cordoba, Spain, ran the EU-funded TOAS project, in which drones were flown over crops to create weed infestation maps for farmers so they could pinpoint which areas to treat.

‘In Europe, 60 % of the pesticides we apply are herbicides to control the weeds,’ said Dr Peña. ‘(With) the technology applied in the crop we can radically reduce the use of these chemicals, and that is a benefit for the environment and the farmer.’

Dr Peña and his team used drones and image analysis technology, which detects slight differences in field colour, to discover vegetation and competitive weeds. However, weeds and budding crops can appear similar in the early season, when crops are at their most vulnerable.

The solution involved examining crop patterns alongside images. ‘We know the crop follows a pattern, and the vegetation that is out of this pattern is classified as weeds,’ said Dr Peña.

‘So in this way we can integrate the spectral information (and) also the position and the shape of the plants in the software to detect those ones that are weeds.’

Farmers can then apply weedkiller to specific areas, monitor the evolution of their crops over time, and create a photorealistic 3D map of their field by taking aerial pictures from multiple views.

Researchers programmed drones to recognise weeds outside normal crop patterns. Image courtesy of TOAS

Maize, olives

After firstly analysing maize fields and olive groves, the team later found the technology worked for other crops such as sunflowers, almonds and grapes.

‘This demonstrates that the technology and the project can go further than the original idea,’ said Dr Peña. ‘As well, we attained a very high accuracy – we can detect more than 95 % of the weeds in the fields. One important thing is that we can detect the parts of the crop field where it’s free of infestation. This place is the part where the farmer does not have to apply the herbicide.’

Spotting the weeds is just one half of the task, though, and researchers elsewhere are developing unmanned ground vehicles, or rovers, that can complete the job by getting rid of the weeds.

“We can radically reduce the use of these chemicals, and that is a benefit for the environment and the farmer.” Dr José M. Peña, Institute for Sustainable Agriculture, Spain.

‘What we are doing is really demonstrating that it can be done fully automatically,’ said Prof. Roland Siegwart of ETH Zurich, Switzerland, who leads an EU-funded robotics project called FLOURISH to develop a prototype of an integrated drone and rover system.

By scanning different crop characteristics, such as height and canopy cover, the drone can communicate the areas that need attention to the unmanned rover on the ground. The rover then removes the weeds, applies pesticide to a specific area, or highlights areas that may need extra fertiliser to the farmer.

Prof. Siegwart says that this could lead to much more efficient use of pesticides.

‘I would estimate it’s probably a very small fraction of a percent of all chemicals which are sprayed on the field which really has an impact,’ he said. ‘If we have more information you can actually, hopefully, get to a much higher efficiency. With probably 100 times less chemistry put on the field you have exactly the same effect.’

Increased research in agricultural technology may also have the added benefit of removing some stigma about drone technology itself as it helps to solve problems such as how to produce enough food for a growing population.

‘Typically we say that around 20-30 % of the whole collection of food on the field is already lost because there are some issues on the field. If you can reduce this it can really help to feed the world’s society.’

Factory-in-a-day

by Fintan Burke

The era of robots that can be controlled by designers rather than engineers will enable Europe’s small- and medium-sized enterprises (SMEs) to compete more effectively against often low-wage competitors in other parts of the world.

‘The human can take the tool, or spindle and guide the robot to the parts and (the robot can) memorise some points or some trajectories,’ explained Dr Dragoljub Surdilovic.

He is part of the EU-funded HEPHESTOS project which developed a prototype of a demonstration-controlled robot that can work with large-scale, hard industrial materials such as metals.

The Issue

In 2014 SMEs provided two out of every three jobs in Europe and 58 % of the value of total goods and services.

As well as supporting technology that will benefit SMEs, the EU’s funding programme Horizon 2020 has reserved approximately EUR 3 billion to directly increase the innovation capacity of smaller businesses whose technologies show commercial potential.

SMEs can also now apply directly for funds to implement high-risk and high-potential ideas that will lead to radical changes in how business is done.

It provides more flexibility than traditional manufacturing machines as SMEs can use only one or a few robots to make their products, regardless of how complex it is to manufacture. If a designer sees the robot making a mistake, they can correct it on the fly.

‘With robots we can apply them for different operations,’ said Dr Surdilovic, who is based at the Fraunhofer Institute for Production Systems and Design Technology in Berlin, Germany. ‘The idea for SMEs was how robots can be used to consider the problems of small-batch production.’

After proving that small-batch production is possible with robots, the next step for the researchers is to improve their design by including new and cheaper sensor technology for future robot units.

This kind of simplified, teachable robots will mean SMEs can avoid outsourcing their work, thereby getting more benefit from their innovations.

Hephestos

‘If a company wants to employ a robot today for doing automation … sometimes it takes days, weeks or even mutiple months,’ said Professor Norbert Krüger of the University of Southern Denmark’s Mærsk Mc-Kinney Møller Institute. ‘That means that if you’re looking to produce a few thousand parts, then it is not a good investment.’

As part of the recently launched RECONCELL project, Prof. Krüger and the other partners in the consortium are working to provide customised robots that can help small businesses quickly set up new production lines, and are taught what to do by demonstration.

‘It’s not only copying, it also needs to understand where the objects are and how they are put together and so on,’ Prof. Krüger said.

Before being deployed in the workshop, the RECONCELL robots rehearse their movements via simulation, fine-tuning their work through virtual trial and error.

‘That’s a key aspect,’ explains Prof Krüger. ‘If you establish a (traditional) robot system, then you do some experiments to find out whether a robot works, this is usually very expensive – particularly when the robot damages something else!’

By the end of the project, in 2018, the aim is to have both the software and hardware at a state such that only few years more are required to arrive at a product for SMEs to build and teach their own production unit.

ReconCell

Same day

While a quick set-up is always the goal with these types of robots, the Factory-in–a-Day project aims to let SMEs start manufacturing on the same day as the robots arrive.

‘Eventually it should be possible to have a small robot installation operational in a day, just like you have your PC operational in a day,’ explained project coordinator Professor Martijn Wisse of Delft University of Technology in the Netherlands.

The project also includes a money-saving element – since the installation period will be dramatically reduced, SMEs can start manufacturing sooner rather than later. This means the time usually spent setting up a production line can now be spent earning an income.

Alongside helping SMEs save time, the project also saves space by including electronic sensors to allow robots and humans to work side-by-side on the floor.

‘If you create a safe robot system that allows the robot to be near a human, or other machines, then you don’t have to create extra safety features that take a lot of time and space,’ explained Prof. Wisse.

Although the project is only a little more than halfway completed, they’ve developed a working prototype of the interface, and were also able to develop the vision software.

It means some companies have already offered to buy the system. ‘It’s not really a matter of getting enough industry involved, it’s more whether we can handle it,’ said Prof. Wisse.

Learn more:

HEPHESTOS 

RECONCELL 

Factory-in–a-Day

Joel Gibbard started out designing prosthetic hands in his bedroom out of sheet metal, but found himself drawn to 3D printing because of its universality and low cost. Now his company, Open Bionics, believes it can cut the cost of a bionic hand by a factor of 20 from the current around EUR 100 000 price tag.

Not only that, but the prosthesis is fully customisable, meaning that reluctant kids can have one themed around characters from the films Iron Man, Frozen and Star Wars.

At the moment, kids don’t really like prosthetics, they’ll take them off, stick them under the bed, and never see them again,’ Gibbard told attendees at the European Robotics Week in Bristol, UK. ‘It makes them feel like a superhero for having (our) prosthetic hand.’

European Robotics Week is a series of events to encourage science, technology engineering and maths education around the theme of robots in society, which is coordinated by euRobotics aisbl, a non-profit organisation which combines around 250 European partners, including industrial groups, universities and users of robotics. It is part of SPARC, the European public-private partnership for robotics in Europe, alongside the European Commission.

Currently the EU-funded robotics programme is the largest civilian-funded robotics research and innovation programme in the world, and its development has been nurtured by the work of the EU-funded coordination project euRobotics and its successor RockEU.

Gibbard, whose company has been helped by a business incubator at the Bristol Robotics Laboratory, a collaborative lab run by the University of Bristol and the University of the West of England, was speaking at an event called From Imagination to Market, where companies and start-ups could explain how they have managed to get their innovations to market.

Grocery

Often companies have benefited from public research funding to help commercialise their innovations, such as that experienced by UK-based online grocery Ocado.

Currently, most of the two million items shipped to customers’ homes every day from Ocado are hand-picked in one of their twelve warehouses, but robotics could make this process more efficient, bringing down the cost of food for consumers.

Ocado engineers have already had success with a simple device – a suction cup mounted on a robotic arm which can pick up thickly packaged items and drop them into a basket. But the suction cup does not fare so well with delicate items such as fruit and vegetables, which can easily be bruised.

That is why Graham Deacon, the team leader of robotics research at Ocado, and his team have been experimenting with more complex robotic claws.

By examining the way humans themselves pick up soft objects – usually, by shaping their fingers around them – he and his colleagues have been trying to teach their robotic claws to do the same as part of an EU-funded research project known as SoMa.

‘We’re trying to get in now, to be part of that solution,’ he said, adding that his company will hopefully have some sort of usable claw to pick delicate items in five to six years.

Tight deadlines

Mike Aldred, the leader of robotics and cordless electronics at Dyson, the UK technology company, believes such timescales are important in business. Whereas in universities researchers tend to think in very long timescales, he said, companies must innovate to relatively tight deadlines.

Aldred gave the example of Dyson’s forthcoming autonomous vacuum cleaner, the Dyson 360 Eye. Once the engineers had established their goal, he said, they put a strict timeframe on getting the product to market.

Another difference between universities and companies, he said, is testing. Whereas academic researchers might be satisfied with a navigation system that works in their clean and tidy laboratory, he said, Dyson has had to test its 360 Eye in 100 000 real households, where it regularly encountered table legs, unruly children and grownups who were not especially tech-savvy.

Even so, Aldred did have some words of encouragement for aspirational roboticists: even if the challenge seems impossible, try it anyway, and you might just succeed. ‘The best person to follow your idea is you,’ he said.

Researchers are developing an exoskeleton prototype that makes it easier to carry heavy loads. Image courtesy of Robo-Mate.

‘We’re getting sufficient power for sufficiently low weight,’ said Jan Veneman, a roboticist at Tecnalia Research and Innovation in Donostia, Spain.

Exoskeletons typically consist of a framework that covers a person’s limbs. The framework has joints alongside the person’s own skeletal joints, and these are powered by motors or springs to assist in strength or endurance.

Exoskeletons with motors are more flexible and more powerful, whereas those with springs can be made much lighter, according Konrad Stadler, an engineer at the Zurich University of Applied Sciences in Switzerland. Stadler is the technical project manager of the Robo-Mate project, and is working on an exoskeleton that could help manual workers. As part of the EU-funded project, he and his colleagues are designing different exoskeleton ‘modules’ to suit the different tasks carried out by manual workers.

In order to help stimulate innovation in manufacturing, the EU has teamed up with industry and academia in public, private partnerships such as the Partnership for Robotics in Europe and Factories of the Future.

Compression forces

‘Musculoskeletal injuries and disorders are caused by lower back compression forces, which result from the load carried and the worker’s own body weight, as well as the posture of the worker,’ says Stadler.

‘Lifting loads with outstretched arms will produce much larger compression forces in the lower back than lifting the same object close to the body.’

By slipping on a Robo-Mate exoskeleton, a worker trying to carry a heavy object with outstretched arms ought to feel more like he or she is carrying it close to the body. Stadler and colleagues are currently testing a prototype in the lab, and hope to be doing industrial tests next year.

Robo-Mate has made a prototype of an exoskeleton. Image courtesy of Robo-Mate.

But exoskeletons are not all about strength. Veneman is investigating whether exoskeletons can also be used to keep people stable as part of the EU-funded BALANCE project.

The human body normally keeps itself balanced with two mechanisms: altering the stiffness of joints, or moving hands and feet to more stable positions. ‘It is very subtle where you place your foot,’ he explained. ‘If you place it too far, or too close, you can easily lose balance.’

The aim of the BALANCE exoskeleton is to learn how the body does this naturally, and see whether the exoskeleton can assist the body to do the same thing. It is currently in the design stage, but Veneman says the prototype should offer minimal resistance to movement, so that it can follow accurately the movements of healthy and infirm people.

Give a push

Once the system understands the human body’s balancing mechanisms sufficiently well, it should be able to replicate it. In theory, you should be able to give someone wearing the BALANCE prototype a push, and the system will move the person’s feet to stop them falling over.

That will require a rather complex exoskeleton with multiple joints, and sensors that can detect instability.

A simpler exoskeleton, and one with a good chance of commercialisation in the near future, is the one being developed within the EU-funded AXO-SUIT project, which is part of the EU’s Active and Assistive Living programme to enhance quality of life for the elderly. The idea behind AXO-SUIT is to help elderly or infirm people perform basic everyday tasks, such as standing up, walking, reaching and grasping.

The technology behind AXO-SUIT is not dissimilar to that in an electric bike, says project lead coordinator Shaoping Bai of Aalborg University in Denmark. Rather than completely taking over, the motors assist action that the user is already trying to perform, such as bending down to water the plants and standing back up again.

AXO-SUIT is currently at the prototype stage. In about five years, however, Bai hopes that his group will have a commercial version costing between EUR 8 000 and EUR 10 000.

That may sound expensive, but Bai believes the exoskeleton could replace a lot of costly personal care for the elderly. ‘It would be helpful for them to maintain their quality of life while staying at home, rather than move to a care centre,’ he said.

More info

Researchers believe robots can be more effective than puppets and other traditional methods of treating autism. Image courtesy of the DREAM project.

Few could deny the power of computer games and avatars to entertain us and perhaps even educate us. But now researchers are turning to internet and robotic technologies to develop new ways of helping children with autism interact with the world around them.

It’s thanks to developments in artificial intelligence that enable computers to read our emotions through our facial expressions, gestures and tone of voice.

Autistic children often struggle to recognise and express emotions, making it difficult to behave appropriately in social situations and leaving them vulnerable to getting bullied at school.

Now, researchers have developed an online game called Camp Exploration, which aims to help autistic children recognise and express emotions.

‘There are a lot of computer games for autistic children, but what makes ours unique is using technology to recognise behaviour and give the children feedback about the way they show emotion,’ says Bjorn Schuller, an Associate Professor at Imperial College London, UK, who led the EU-funded ASC-Inclusion project which developed the game.

To progress through Camp Exploration, players have to interact through facial expressions, body gestures and vocalisations to express emotions, which are picked up via a webcam and microphone. The intelligent software can then give the player feedback about their communication.

Testing the game in an open trial involving high-functioning autistic children showed not only that they enjoyed playing it, but also that there were significant improvements in the children’s ability to show their emotions and social recognition.

Camp Exploration is being commercialised and should be widely available soon. ‘The technology is definitely ready,’ said Prof. Schuller. ‘We hope it will help the autistic community access therapy, as professional care therapists and specialist schools are not available for autistic children the world over.’

Therapy
When therapists do work with autistic children, they often use puppets and animated characters to engage them in interactive play. However, researchers believe that small, friendly looking robots could be even more effective, not just to act as a go-between, but because they can learn how to respond to a child’s emotional state and infer his or her intentions.

‘We hope it will help the autistic community access therapy.’

Professor Bjorn Schuller, Imperial College London, UK

‘Children with autistic spectrum disorders prefer to interact with non-human agents, and robots are simpler and more predictive than humans, so can serve as an intermediate step for developing better human-to-human interaction,’ said Professor Bram Vanderborght of Vrije Universiteit Brussel, Belgium.

He is part of the EU-funded DREAM project, which is developing robots that can operate autonomously for limited periods of time under the supervision of a therapist and decipher emotions to help autistic children interact in a social environment.

The researchers are programming the robots to contain software to detect where the children are looking during the session, which enables them to evaluate a child’s level of engagement and automatically adapt their behaviour to grab attention, if necessary. Quantifying the level of engagement through markers like these will also help therapists to measure a child’s progress more objectively than is currently possible.

‘Researchers have found that children with autism spectrum disorders are more responsive to social feedback when it is provided by technological means, rather than a human,’ said Prof. Vanderborght.

The ethical and legal considerations of using robots to interact with children are enormous: there are fears about safety, trust, and who should take responsibility for the robot’s behaviour. These considerations are being evaluated hand-in-hand, as DREAM reserachers are developing and testing the technology within an integrated research programme.

‘DREAM is paving the way, not only for other forms of robot-enhanced therapy, but also for more integration between research and innovation and ethics,’ said Professor Mark Coeckelbergh of De Montfort University, UK, who is also working on the project.

Combatting social disorders
Humanoid robotics and computer avatars could also help rehabilitate people suffering from social disorders such as schizophrenia or social phobia. It’s thanks to the theory of similarity, which suggests that it is easier to interact socially with someone who looks, behaves or moves like us.

Researchers on the EU-funded AlterEgo project have developed a system to enable a robot or computer avatar to interact with a patient whilst playing a version of the mirror game, in which two players try to copy each other’s motion whilst playing with coloured balls that can move horizontally on a string.

Initially the avatar is like an alter ego, created to look and move like the patient to enhance his or her feelings of attachment. Over time the avatar is slowly altered to become less similar, thus helping with social rehabilitation.

‘It is very challenging to build an avatar that is intelligent enough to synchronise its motion with a human player, but our initial results are very exciting,’ says Professor Mario di Bernardo, from the University of Bristol, UK. He says the researchers are now looking at getting groups of people and avatars interacting with one another, setting up multiple human-machine interactions for social rehabilitation.

• DREAM
• ASC-Inclusion
• AlterEgo.