Robohub.org
 

Are gestures the future of robotic control?

by
29 April 2017



share this:

A few decades ago, touchscreens were impressive, yet clunky pieces of technology reserved for applications that did little more than show off that touchscreens were possible. Today’s touchscreens are commonplace and readily accepted as an easy way to interact with an operating system; they’re accurate, support multi-touch, are comfortable to use, and can even operate in multiple dimensions (pressing hard).

We may not have perfected touchscreens yet, but we’re getting close. Accordingly, engineers and researchers are already speculating about the next phase of UI development, especially for robotics control. So far, the leading candidate is gesture-based control—the use of physical gestures to relay commands.


The idea

The major limitation for touchscreens is the fact that they operate only in two dimensions; the third dimension introduced with force touch is extremely limited. Comparatively, hand gestures and physical movements can operate in three dimensions, and depending on how they’re designed, could feel more natural than swiping and tapping a smartphone screen.

Demand for three-dimensional gestural control is increasing with the onset of virtual reality (VR) and augmented reality (AR) technology; because the digital world we experience will be moving from two dimensions to three dimensions, the controls we use to manage those experiences will also need to change. With enough sophistication, these systems could provide better feedback to the users in control; rather than merely responding with visual feedback like movement or lights, users could feasibly be immersed with physical feedback like vibration or resistance.


Where we stand

Currently, one of the most advanced gestural systems is the Real-Time 3D Gesture Analysis for Natural Interaction with Smart Devices, a project headed by researchers in Linnaeus University in Sweden. However, simpler forms of gesture-based control are already available.

For example, there are projects that use Arduino to create a robot that can respond to four different simple hand gestures (plus a neutral position). Of course, iPhone technology also makes use of some simple “gestures,” such as shaking the phone to undo typing or rotating the phone to experience an immersive 360-degree view of an image or video.


The main obstacles

There are a few obstacles preventing gestures from being solidified as the next landmark in robotics control, however:

  • Gesture complexity. The biggest obstacle described by researchers at Linnaeus University is the sheer complexity of physical gestures; to be effective, their recognition systems need to be able to gather and recognize thousands of tiny data points and interpret complex patterns to “understand” what movement is being attempted. Moreover, this big data-based interpretation needs to happen in real-time—especially for applications like live robotics control, or engagement in a VR-based video game. That demands not only an incredibly intelligent system, but a processor that can operate quickly.
  • Accessibility. Gesture recognition systems would likely be developed to accommodate a “standard” human model, such as a “standard” right human hand. How would the system accommodate somebody whose hand was missing, or who is missing a few fingers? What about left-handed people? And people with Parkinson’s disease, or who are unable to operate their hands with precision control?
  • Applications. For gestures to be accepted as a mainstream way to interact with robotics and virtual environments, robotics and virtual environments need to be available. Virtual reality (VR) technology has been advancing strongly for the past several years and is poised to take a big step forward in user adoption by 2020, but until those user adoption numbers are reliable, the demand for gestural systems is relatively low. Using hand movements and other physical gestures to control a two-dimensional screen, for example, would be largely ineffective.

On paper, gestures seem like the best form of control for the digital devices, robotics, and VR systems of the future. However, there are still many obstacles to overcome before we’re ready for large-scale adoption. Fortunately, researchers are ahead of the curve, already preparing the intelligent gesture-based recognition programs we’ll need when touchscreens become obsolete.



tags: , , , , , ,


Anna Johansson is a freelance writer, researcher, and business consultant.
Anna Johansson is a freelance writer, researcher, and business consultant.





Related posts :



Robot Talk Episode 34 – Interview with Sabine Hauert

In this week's episode of the Robot Talk podcast, host Claire Asher chatted to Dr Sabine Hauert from the University of Bristol all about swarm robotics, nanorobots, and environmental monitoring.
28 January 2023, by

Special drone collects environmental DNA from trees

Researchers at ETH Zurich and the Swiss Federal research institute WSL have developed a flying device that can land on tree branches to take samples. This opens up a new dimension for scientists previously reserved for biodiversity researchers.
27 January 2023, by

The robots of CES 2023

Robots were on the main expo floor at CES this year, and these weren’t just cool robots for marketing purposes. I’ve been tracking robots at CES for more than 10 years, watching the transition from robot toys to real robots.
25 January 2023, by

Robot Talk Episode 33 – Interview with Dan Stoyanov

In this week's episode of the Robot Talk podcast, host Claire Asher chatted to Professor Dan Stoyanov from University College London all about robotic vision, surgical robotics, and artificial intelligence.
20 January 2023, by

When a professor meets a farmer

There's a clear need for technology in farming but its problems are systemic, finds a critical design professor when she visits a flower bulb farmer. Watch this new episode of FRAIM in the Field.
19 January 2023, by





©2021 - ROBOTS Association


 












©2021 - ROBOTS Association