A few decades ago, touchscreens were impressive, yet clunky pieces of technology reserved for applications that did little more than show off that touchscreens were possible. Today’s touchscreens are commonplace and readily accepted as an easy way to interact with an operating system; they’re accurate, support multi-touch, are comfortable to use, and can even operate in multiple dimensions (pressing hard).
We may not have perfected touchscreens yet, but we’re getting close. Accordingly, engineers and researchers are already speculating about the next phase of UI development, especially for robotics control. So far, the leading candidate is gesture-based control—the use of physical gestures to relay commands.
The major limitation for touchscreens is the fact that they operate only in two dimensions; the third dimension introduced with force touch is extremely limited. Comparatively, hand gestures and physical movements can operate in three dimensions, and depending on how they’re designed, could feel more natural than swiping and tapping a smartphone screen.
Demand for three-dimensional gestural control is increasing with the onset of virtual reality (VR) and augmented reality (AR) technology; because the digital world we experience will be moving from two dimensions to three dimensions, the controls we use to manage those experiences will also need to change. With enough sophistication, these systems could provide better feedback to the users in control; rather than merely responding with visual feedback like movement or lights, users could feasibly be immersed with physical feedback like vibration or resistance.
Currently, one of the most advanced gestural systems is the Real-Time 3D Gesture Analysis for Natural Interaction with Smart Devices, a project headed by researchers in Linnaeus University in Sweden. However, simpler forms of gesture-based control are already available.
For example, there are projects that use Arduino to create a robot that can respond to four different simple hand gestures (plus a neutral position). Of course, iPhone technology also makes use of some simple “gestures,” such as shaking the phone to undo typing or rotating the phone to experience an immersive 360-degree view of an image or video.
There are a few obstacles preventing gestures from being solidified as the next landmark in robotics control, however:
On paper, gestures seem like the best form of control for the digital devices, robotics, and VR systems of the future. However, there are still many obstacles to overcome before we’re ready for large-scale adoption. Fortunately, researchers are ahead of the curve, already preparing the intelligent gesture-based recognition programs we’ll need when touchscreens become obsolete.