By Eran Klein, University of Washington and Katherine Pratt, University of Washington

In the 1995 film “Batman Forever,” the Riddler used 3-D television to secretly access viewers’ most personal thoughts in his hunt for Batman’s true identity. By 2011, the metrics company Nielsen had acquired Neurofocus and had created a “consumer neuroscience” division that uses integrated conscious and unconscious data to track customer decision-making habits. What was once a nefarious scheme in a Hollywood blockbuster seems poised to become a reality.

Recent announcements by Elon Musk and Facebook about brain-computer interface (BCI) technology are just the latest headlines in an ongoing science-fiction-becomes-reality story.

BCIs use brain signals to control objects in the outside world. They’re a potentially world-changing innovation – imagine being paralyzed but able to “reach” for something with a prosthetic arm just by thinking about it. But the revolutionary technology also raises concerns. Here at the University of Washington’s Center for Sensorimotor Neural Engineering (CSNE) we and our colleagues are researching BCI technology – and a crucial part of that includes working on issues such as neuroethics and neural security. Ethicists and engineers are working together to understand and quantify risks and develop ways to protect the public now.

Picking up on P300 signals

All BCI technology relies on being able to collect information from a brain that a device can then use or act on in some way. There are numerous places from which signals can be recorded, as well as infinite ways the data can be analyzed, so there are many possibilities for how a BCI can be used.

Some BCI researchers zero in on one particular kind of regularly occurring brain signal that alerts us to important changes in our environment. Neuroscientists call these signals “event-related potentials.” In the lab, they help us identify a reaction to a stimulus.

In particular, we capitalize on one of these specific signals, called the P300. It’s a positive peak of electricity that occurs toward the back of the head about 300 milliseconds after the stimulus is shown. The P300 alerts the rest of your brain to an “oddball” that stands out from the rest of what’s around you.

For example, you don’t stop and stare at each person’s face when you’re searching for your friend at the park. Instead, if we were recording your brain signals as you scanned the crowd, there would be a detectable P300 response when you saw someone who could be your friend. The P300 carries an unconscious message alerting you to something important that deserves attention. These signals are part of a still unknown brain pathway that aids in detection and focusing attention.

Reading your mind using P300s

P300s reliably occur any time you notice something rare or disjointed, like when you find the shirt you were looking for in your closet or your car in a parking lot. Researchers can use the P300 in an experimental setting to determine what is important or relevant to you. That’s led to the creation of devices like spellers that allow paralyzed individuals to type using their thoughts, one character at a time.

It also can be used to determine what you know, in what’s called a “guilty knowledge test.” In the lab, subjects are asked to choose an item to “steal” or hide, and are then shown many images repeatedly of both unrelated and related items. For instance, subjects choose between a watch and a necklace, and are then shown typical items from a jewelry box; a P300 appears when the subject is presented with the image of the item he took.

Everyone’s P300 is unique. In order to know what they’re looking for, researchers need “training” data. These are previously obtained brain signal recordings that researchers are confident contain P300s; they’re then used to calibrate the system. Since the test measures an unconscious neural signal that you don’t even know you have, can you fool it? Maybe, if you know that you’re being probed and what the stimuli are.

Techniques like these are still considered unreliable and unproven, and thus U.S. courts have resisted admitting P300 data as evidence.

Imagine that instead of using a P300 signal to solve the mystery of a “stolen” item in the lab, someone used this technology to extract information about what month you were born or which bank you use – without your telling them. Our research group has collected data suggesting this is possible. Just using an individual’s brain activity – specifically, their P300 response – we could determine a subject’s preferences for things like favorite coffee brand or favorite sports.

But we could do it only when subject-specific training data were available. What if we could figure out someone’s preferences without previous knowledge of their brain signal patterns? Without the need for training, users could simply put on a device and go, skipping the step of loading a personal training profile or spending time in calibration. Research on trained and untrained devices is the subject of continuing experiments at the University of Washington and elsewhere.

It’s when the technology is able to “read” someone’s mind who isn’t actively cooperating that ethical issues become particularly pressing. After all, we willingly trade bits of our privacy all the time – when we open our mouths to have conversations or use GPS devices that allow companies to collect data about us. But in these cases we consent to sharing what’s in our minds. The difference with next-generation P300 technology under development is that the protection consent gives us may get bypassed altogether.

What if it’s possible to decode what you’re thinking or planning without you even knowing? Will you feel violated? Will you feel a loss of control? Privacy implications may be wide-ranging. Maybe advertisers could know your preferred brands and send you personalized ads – which may be convenient or creepy. Or maybe malicious entities could determine where you bank and your account’s PIN – which would be alarming.

With great power comes great responsibility

The potential ability to determine individuals’ preferences and personal information using their own brain signals has spawned a number of difficult but pressing questions: Should we be able to keep our neural signals private? That is, should neural security be a human right? How do we adequately protect and store all the neural data being recorded for research, and soon for leisure? How do consumers know if any protective or anonymization measures are being made with their neural data? As of now, neural data collected for commercial uses are not subject to the same legal protections covering biomedical research or health care. Should neural data be treated differently?

These are the kinds of conundrums that are best addressed by neural engineers and ethicists working together. Putting ethicists in labs alongside engineers – as we have done at the CSNE – is one way to ensure that privacy and security risks of neurotechnology, as well as other ethically important issues, are an active part of the research process instead of an afterthought. For instance, Tim Brown, an ethicist at the CSNE, is “housed” within a neural engineering research lab, allowing him to have daily conversations with researchers about ethical concerns. He’s also easily able to interact with – and, in fact, interview – research subjects about their ethical concerns about brain research.

There are important ethical and legal lessons to be drawn about technology and privacy from other areas, such as genetics and neuromarketing. But there seems to be something important and different about reading neural data. They’re more intimately connected to the mind and who we take ourselves to be. As such, ethical issues raised by BCI demand special attention.

Working on ethics while tech’s in its infancy

As we wrestle with how to address these privacy and security issues, there are two features of current P300 technology that will buy us time.

First, most commercial devices available use dry electrodes, which rely solely on skin contact to conduct electrical signals. This technology is prone to a low signal-to-noise ratio, meaning that we can extract only relatively basic forms of information from users. The brain signals we record are known to be highly variable (even for the same person) due to things like electrode movement and the constantly changing nature of brain signals themselves. Second, electrodes are not always in ideal locations to record.

All together, this inherent lack of reliability means that BCI devices are not nearly as ubiquitous today as they may be in the future. As electrode hardware and signal processing continue to improve, it will be easier to continuously use devices like these, and make it easier to extract personal information from an unknowing individual as well. The safest advice would be to not use these devices at all.

The goal should be that the ethical standards and the technology will mature together to ensure future BCI users are confident their privacy is being protected as they use these kinds of devices. It’s a rare opportunity for scientists, engineers, ethicists and eventually regulators to work together to create even better products than were originally dreamed of in science fiction.

Research and development of robotic assistive technologies has gained tremendous momentum in the last decade due to several factors such as the maturity level reached by several technologies, the advances in robotics and AI and the fact that more than 700 million of persons have some kind of disability or handicap. For many people with mobility impairments, essential and simple tasks, such as dressing or feeding, require the assistance of dedicated people. Thus, the use of devices providing independent mobility can have a large impact on their quality of life.

A team of researchers from the University of Cassino, under the direction of Professor Filippo Arrichiello, is working on the development of a control architecture for assistive robotic systems operated via a Brain-Computer Interface (BCI) to allow a user to operate a robot using his thoughts. The proposed software architecture relies on widely used frameworks to operate BCIs and robots (namely, BCI2000 for the operation of the BCI and ROS for the control of the robot) integrating control, perception and communication modules developed for the application at hand.

In particular, using the BCI2000 framework, they developed a Graphical User Interface (GUI) based on P300 paradigm to allow the user to select options via a BCI Epoch+. The user can select objects from the scene and the action to perform by focusing attention on the corresponding flashing icons on a screen. By counting how many times the icon he wants to select flashes, the user generates a P300 potential that is detected via the BCI and translated in a selection operation on the GUI.

Messages related to the object and actions selected via the GUI are automatically sent to a 7DOF lightweight robot manipulator Jaco2 that has to operate in the environment. The robot control architecture relies on a perception module that allows to detection and localization of objects, and the face/mouth of the user, using a RGB-D sensor (Kinect one). The motion of the manipulator is then controlled relying on a closed loop inverse kinematic algorithm that simultaneously manages multiple setbased and equality-based tasks. Preliminary experiments have been performed where the user commands the robot via the BCI in order to move objects on a table or to select a bottle from where to drink.

Team members: prof. Filippo Arrichiello, Paolo Di Lillo (PhD student), Daniele Di Vito (PhD student), prof. Gianluca Antonelli, prof. Stefano Chiaverini.

Click here for more information on the Robotics Research Group of the DIEI.

If you enjoyed this article, you might also be interested in:

• Virtual race: Competing in Brain Computer Interface at Cybathlon
• Brain Computer Interface used to control the movement and actions of an android robot
• Can a brain-computer interface convert your thoughts to text?

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Credit: ETH Zurich / Alessandro Della Bella

Millions of people worldwide rely on orthotics, prosthetics, wheelchairs and other assistive devices to improve their quality of life. In the United States alone, there are more than 1.6 million people with limb amputations. The World Health Organization estimates the number of wheelchair users to be about 65 million people worldwide.

It is important to improve the daily lives of people with disabilities or physical weaknesses, and allow them to be more independent. Unfortunately, current assistive technology does not fully address their needs. Wheelchairs cannot climb stairs; arm prostheses do not enable versatile hand functions. Powered support devices have limited battery life. The list goes on and on.

Many people need help climbing stairs. Credit: ETH Zurich/Alessandro Della Bella.

People with disabilities are often disappointed with their devices’ performance, and choose not to use them. The main objection is that designs ignore user needs, such as individual preferences for device appearance, sizing and fitting for comfort and function, effort and time required to put on and take off, and device durability and weight.

Beyond the design issues, these tools are expensive. Cost shuts many people out of using them, regardless of how well they work.“ And stairs, steep ramps, narrow doorways and low tables can make the use of assistive technologies very cumbersome or even impossible.

It is an industry ripe for innovation. To encourage this work, I have founded a new kind of competition promoting the development of useful technologies. In the Paralympics, parathletes aim to achieve maximum performance in sporting challenges. In our new contest, the Cybathlon, people with physical disabilities will compete against each other at tasks of daily life, with the aid of advanced assistive devices – including robotic ones.

Focusing on teamwork and technology

In the Cybathlon, what’s being tested is not just the abilities of human athletes, nor only the equipment they use. Rather, it’s their symbiosis, balancing good technical performance of the device, and its control by the athlete.

Competitors will face off in six disciplines, for people with either limb amputations or limb paralysis of varying degrees, such as occurs after a spinal cord injury. We’ll organize a race focused on each of these technologies: powered leg prostheses, powered arm prostheses, functional electrical stimulation (FES) driven bikes, powered wheelchairs, and powered exoskeletons. The sixth competition is a racing game with virtual avatars controlled by brain-computer interfaces.

Teams work together: a pilot in control, with others supporting and operating the technology. Credit: ETH Zurich/Alessandro Della Bella.

We ran test sessions in July 2015, and have slated the full competition for October 8 in Zurich. The devices involved can be prototypes developed by research labs or companies, or commercially available products. Competitors will be called pilots, as they must control a device that enhances their mobility.

Competing teams each consist of a pilot, scientists and technology providers, making the Cybathlon also a competition among companies and research laboratories. As a result there are two awards for each competition’s winning team: a medal for the pilot and a cup for the company or lab that made the device.

The six competitions

The competitions will simulate challenges people with disabilities face in daily life – situations that non-disabled people don’t think twice about but that can be insurmountable for others.

Powered prosthetic legs Most leg prostheses require their users to swing the artificial leg just so, to properly align the knee, lower leg and foot. And they cannot transfer muscular power through the knee, using thigh muscles to help climb stairs, for example.

https://www.youtube.com/watch?v=4klZXwE3-tI

One of the best unpowered-knee prostheses, offering a quite symmetrical gait compared to most devices.

Powered leg prostheses can provide that missing power, but they are difficult to control unless the motor understands how the user wants to move. And even the best batteries are either too heavy or too short-lived to be a real solution. Our race will challenge pilots with above-knee amputations to use powered prosthetic legs to walk up and down stairs, stand up from a seated position, and otherwise navigate a complicated environment.

Powered hands and arms Two-handed jobs, requiring either strength (like carrying a heavy box) or specific fine motor skills (like opening a small jar of jam) are challenging with even the best upper-arm prostheses. As a result, up to 60 percent of people with upper-limb amputation don’t use their prosthetic device very much or even at all. People are much less likely to reject more advanced devices, like body-powered or electric ones. Pilots with amputations at or above the lower arm will use motorized prosthetic hands and arms to complete various household and food preparation tasks.

Assisted cycling People with complete paraplegia don’t lose their muscles; they just lose the ability to control them. A technology called functional electrical stimulation (FES) can restore some of this function, sending electricity into otherwise dormant nerves to activate muscles. FES technology has been used for decades. But the systems take a long time to set up, don’t produce much muscle force, and tire out muscles quickly.

Surgically implanted systems give more specific control of particular muscles and higher force output. But they are expensive and invasive, and carry more risks than external FES devices that are merely strapped to a person’s body. For these reasons, doctors and patients don’t often use FES technology.

In the Cybathlon, pilots with complete paraplegia will compete in a bike race, using FES devices to fire their leg muscles to drive the pedals.

Helping cyclists pedal with functional electrical stimulation.

Powered wheelchairs Despite the Americans with Disabilities Act and other laws and regulations, public buildings are still hard to enter and navigate in wheelchairs. Most outdoor devices are too bulky and not agile enough for indoor use; commercial indoor wheelchairs can’t travel over uneven terrain or steps. So-called intelligent or smart wheelchairs, which can autonomously navigate in known environments have been available for decades, but are very expensive and used by relatively few people.

Wheelchairs are becoming more powerful, but often their control systems are neither as effective nor as comfortable as they could be. To push development of these functions, pilots with paralysis will take powered wheelchairs through an obstacle course with ramps, stairs, bends, doors and uneven terrain.

Powered wheelchair racing.

Powered exoskeletons An alternative to wheelchairs are exoskeletal devices that help people walk. However, batteries only last a few hours, and the equipment is very bulky and heavy. Most of the commercially available multi-joint exoskeletons weigh between 46 and 62 pounds (21–28 kg). One device, called “REX” weighs nearly 88 pounds!

Current commercial systems are so limited that they can’t even climb slopes or stairs. An obstacle course will test pilots’ and teams’ abilities to develop systems that can move through difficult areas.

Moving with powered exoskeletons.

Navigating by brain power In the brain-computer interface (BCI) race, pilots with paralysis of all four limbs will control a virtual avatar in a racing game displayed on a computer screen. The best pilots will be able to make their brain signals emit three different commands to overcome three different kinds of virtual obstacles. BCI technology is becoming more popular, but most systems take a long time to set up, can be uncomfortable, and don’t function well outside the lab. That has prevented its broad use in daily life.

Racing with brain power alone.

Pushing the boundaries of the possible

The Cybathlon will bring together people with disabilities or physical weaknesses, researchers and developers, governments and other agencies that fund services and research. It will also showcase the importance of this work to the general public. Our hope is that over time, these devices will become more affordable and more functional.

Unlike the Paralympic Games, pilots can use any technical aids they need, as long as they are safe. That enables people with more severe disabilities to compete. The goal is not to be the fastest or the strongest participant; rather it’s to be the most skilled pilot who can use advanced technologies to best overcome the challenges of everyday life.

This article was originally published on The Conversation. Read the original article.

If you liked this article, you may also want to read:

• Tickets for Cybathlon now on sale
• New video shows bionic athletes rehearsing for upcoming Cybathlon competition
• Exoskeletons: From helping people walk to controlling robots in space
• Body-enhancing exoskeletons could be stepping into industrial trials next year
• Are skintight suits the future of robotic exoskeletons?
• Robots Podcast #215: Human 2.0: Exoskeletons and Orthoses, with Hugh Herr

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

For the very first time, the 6th International Brain-Computer Interface (BCI) Meeting Series will offer free remote attendance, via live-stream.

Livestream will be available today, Monday 30 May, during the evening opening session at 19:30 PST, and tomorrow, Tuesday 31 May, during morning 9:30 PST and afternoon sessions, 13:30 and 14:30 PST respectively.

The BCI Meeting will open with the Once and Future BCI Session, featuring speakers: Eberhard Fetz, Emanuel Donchin, and Jonathan Wolpaw.

Tuesday morning will contain the State of BCI Symposium, with speakers: Nick Ramsey, Lee Miller, Donatella Mattia, Aaron Batista, and José del R. Millán. Tuesday afternoon will be the Virtual Forum of BCI Users and Selected Oral Presentations. The remainder of the BCI Meeting are poster sessions and workshops and cannot be experienced remotely.

You can read all papers submitted to the BCI meeting here.

Please pass the livestreaming link (http://bcisociety.org/livestream/) to anyone else who may be interested in remote participation at the 2016 BCI Meeting.

Livestreaming link

Our immediate goal is to develop novel, integrated technologies to support independent limb repositioning, which is particularly valuable for those with reduced muscular strength, and helps minimize the occurrence of contractures and pressure ulcers. Our longer term goal is to develop devices that can also be used for assisting such individuals with personal care tasks, and for transferring themselves, from a bed to a commode, for example.

System requirements

The most prominent challenges include accessible and multimodal interaction, autonomy, safety, privacy, affordability, real-time performance, and intelligence.

There are a variety of assistive technologies for input control, and depending on the nature of their disability, some users may prefer a trackball mouse while others prefer speech recognition or an eye-tracking system. Another important design consideration is to determine when it is permissible for robots to override user commands or not follow instructions. Since the capabilities of robotic devices exceed the speed and force of human movement, safety is of great concern; a user may not be able to respond quickly and effectively when the robot behaves unexpectedly.

Because personal care robots are equipped with the ability to sense, process and record information, their use raises concerns about client privacy. Solutions must also be affordable, and strike a balance between price and capability if they are to be successfully marketed in the long run.

In addition, being able to provide immediate assistance is critical when urgent medical care is needed, so the assistive robot must be able to perform tasks in real-time. It must be sensitive and intelligent, delivering services while ensuring that it does not inadvertently cause harm or injury to its users. Such a device must be able to learn and intuitively make decisions in order to safely trade off user comfort and speed when required.

UMBC Research

At the University of Maryland, Baltimore County (UMBC), we recently began developing a solution to support the movement and repositioning of an individual’s arms. We first conducted a survey of people with severely limited arm strength. The aim was to examine their attitudes toward using BCI and speech recognition tools, such as the Emotiv Epoc headset pictured below, for controlling a robotic aid to assist with arm repositioning. The Emotiv headset is an off-the-shelf electroencephalography (EEG) system that provides a non-invasive means of recording electrical activity of the brain along the scalp, and can detect emotions, facial expressions, and conscious thoughts.

Emotiv Epoc BCI Headset. Source: Emotiv

From our study we learned that participants were most interested in multimodal interaction since it provides the most options for effective control. The preferred interaction mode was speech recognition for the overall tasks. However, all the participants said that a BCI device was also useful when speech recognition was not possible, such as when brushing teeth or when breathing difficulties make speech-input solutions challenging. We therefor decided to integrate BCI and speech recognition.

To show the feasibility of integrating BCI with speech recognition for self-directed arm repositioning tasks, we designed a robotic interface for repositioning the simulated arm of an avatar using the Emotiv Epoc headset and Dragon NaturallySpeaking voice recognition software. The system incorporated an accessible GUI.

BCI robot interface with female avatar. Source: UMBC

In an exploratory pilot study, four participants without disabilities performed ten arm movement tasks using four inputs (the keyboard, Emotiv Epoc headset, mouse, and Dragon NaturallySpeaking) in sequential order to move the avatar’s shoulder and elbow joints up and down. Participants used a motion imagery paradigm to control the avatar, for example looking towards the left to raise the avatar’s left elbow, looking towards the right to raise the avatar’s right elbow, and so on.

Total time spend using all four interaction methods. Source: UMBC

Preliminary results suggest that, while all participants were able to interact with the avatar using all four input methods, using BCI and speech were most challenging. However, time spent training with Dragon NaturallySpeaking and the Emotiv Epoc can improve performance, and it should be noted that individuals with severely limited hand strength may find voice control and BCI relatively easier to use than a keyboard and mouse.

Further work at UMBC has focused on developing designs for three robotic transferring prototypes, a powered commode wheelchair, a toileting aid, a robotic toothbrush system, and a universal gripper for feeding. Videos simulations of these prototypes can be watched in the playlist below. Kavita Krishnaswamy’s PhD proposal defense Increased Autonomy with Robotics for Daily Living further discusses effective assistive prototypes.

Outlook

One of the most craved aspects of the human experience is to be independent: the abilitiy to take care of one’s self establishes a sense of dignity, inherent freedom, and profound independence. Our goal is to bring robotic assistive devices into the real world where they can support individuals with severe disabilities and alleviate the workload of caregivers, with the ultimate vision of helping people with severe physical disabilities to achieve physical independence without relying on others. As robotic assistive devices become ubiquitous, they will enable people with severe physical disabilities to confidently use technology in their daily lives, not just to survive, but to flourish.

• Unpowered exoskeleton improves efficiency of human walking
• Noonee testing Chairless Chair exoskeleton at Audi production plants
• Walking assistive devices for the elderly
• Readers optimistic about role of robots as care assistants for seniors

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Brain-computer interfacing (BCI) is a rapidly growing field that offers huge potential for many applications, such as medical grade BCIs to help people with sensory-motor disabilities. Currently, a number of researchers are developing more affordable BCI systems designed to address a wider range of neurotherapeutic applications.

Conor Russomanno is the founder and research developer of OpenBCI, a low-cost open-source hardware platform that records the brain’s electrical signals and uses devices and software languages to make the data easily accessible. Russomanno and co-founder Joel Murphy aim to accelerate the advancement of BCI through collaborative hardware and software development.

The OpenBCI project began earlier this year after raising over $200K through a Kickstarter campaign (more than doubling their initial goal). Even in the early stages, OpenBCI is already drawing a lot of attention and is having interesting repercussions in the BCI world.

“There’s a lot of really exciting stuff that’s going to happen in the near future and in the not so distant future, but I think the one thing that we’re hoping to achieve with open BCI is to really lower the barrier of entry both in terms of educational materials and also cost,” says Russomanno.

http://youtu.be/a55_EfFtysc

A number of consumer-grade EEG devices that have a relatively low channel capacity are available, but higher quality, medical and research grade EEG systems are much more expensive than a standard OpenBCI kit, which makes them less accessible. One of the advantages of Russomanno’s and Murphy’s development is that whereas previously, one medical grade system was used in a classroom or research laboratory, now, the same amount of money can be used to acquire  enough OpenBCI systems to provide every student with their own device.

“This is reflected in our consumer base. A huge proportion of our customers are students, graduate-level researchers and professors who want to use OpenBCI as a learning tool, explains Russomanno.

Another of OpenBCI’s exciting potentials is the fact that it is fully open source. Russomanno and Murphy are using a lot of their resources to create an infrastructure in which people are not just able to use the technology but can also share their experiences and connect with other people using the technology. This enables users to share tutorials, getting started guides and also their own individual projects. In this sense, the creators have not only provided a tool, they are also establishing a level of enthusiasm for an unprecedented unicentric development.

The fact that such enthusiasm is being drummed up in many sectors from academia to the business world, marks a big step for BCI. How far into the mainstream does OpenBCI have the potential to go?

“I think that to be realistic about the technology it’s important to realize that, especially in the commercial space, technology like this has a tendency to get mislabelled and misrepresented by media sources. When you see the catchphrase ‘mind controlled robot’ you think of someone sitting in a chair with electrodes plugged into their head having ultimate control over many, many dimensions of a robot’s functionality.” Says Russomanno.

In reality, however, in the current state, mind controlled robots, at least using OpenBCI, translates to looking at a flashing screen and having the frequency of the flashing induce a similar frequency in the brain, and then telling the robot to turn left.

“We’re are many steps away from wearing a device that knows what turning your car on, versus opening your door look like. So it’s important to be realistic about what the technology is capable of,” Russomanno explains.

Russomanno points out that for BCI to advance, it’s essential for people to work together. Although there is some competition within this space, he emphasizes the importance of cooperation. Progress from any one company is going to help the industry as a whole. People in tested in joining the OpenBCI community can buy the technology, make use of the many available resources and share their experiences and this will work toward pushing BCI forward into the future.

Image credit: 4bpblogspot.com

The BrainGate Group is on the cutting edge of an emerging technology – brain-machine interfaces. Their advances have allowed individuals with extensive paralysis and locked-in syndrome to interact with the world around them by controlling a computer mouse with their mind. Moving a mouse is something that most of us take for granted, but an individual with locked-in syndrome, that is fully awake and conscious, has no other means of producing speech, move limbs, or make facial movements.

Through a device implanted in the skull, the BrainGate device interfaces with a computer, allowing the individual to interact with their environment. This technology is made possible in part by extensive mapping of the neural activity of the brain, but revolutionary hardware designed by engineers like Bacher have also been key.

Bacher began working on the BrainGate technology by conducting brain-machine interface research on non-human primates. Or, as Bacher describes it, “basically having monkeys playing video games with their brains.” This research helped Bacher and other researchers understand how the brain works and begin development of the assistive technologies. When the opportunity to join the BrainGate group presented itself, Bacher jumped at the chance.

Says Bacher, “Although I’m interested in the basic neuroscience and love the pure technology aspects of it, thinking about how this translates to improve the quality of life for people in the real world has always been the underlying motivation for me.”

Outside of his work on BrainGate, Bacher is passionate about connecting disabled individuals with assistive technologies. This passion led Bacher to found the Speak Your Mind Foundation, which creates, distributes, and supports assistive communication technologies for individuals who are unable to effectively communicate due to neurological injury and disease. Says Bacher, “there is still a huge gape between technology creators and healthcare providers.” Bacher hopes to help bridge this gap and bring these technologies to the healthcare market.

Bacher’s approach also takes worldwide emerging technology trends in hands-free computing, robotics, and other areas, and finds ways to translate them into helping disabled individuals. Using technologies developed for larger markets helps lower the costs in developing applications for assistive technologies.  While the technological solutions are currently tailored for specific individuals involved in clinical trials, the solutions developed for them have potential applications for larger groups.

Says Bacher, “if we develop a piece of technology for someone with a spinal cord injury who is moving their head to control a computer, there’s no reason that can’t help someone whose suffered a stroke.”

Human-computer interface technology is still largely limited to touch, keyboard, and mouse. The research conducted by Bacher and those like him has the potential to change this, and as these technologies progress, their potential applications widen.

Says Bacher, “clearly, a mouse and a keyboard, even today seems like an antiquated way to interact with technology. We’ve seen developments like touch-screen technology and voice-control come along way, but it’s still not mainstream. In order to really penetrate the commercial marketplace, there’s a high threshold for how well something has to work.”

While current research is focused on assistive technologies for disabled individuals, the possible applications don’t end there. Part of the reason for this is the “symbiotic” relationship developing between consumer and assistive technologies. Smart phones can provide the “brains” for assistive devices. Eye motion-capture technologies developed for paralyzed individuals open up new ways for the general public to interact with technology, and hands free technologies benefit individuals without the use of their arms. Bacher believes that technologies like Google Glass, Leap Motion, Kinect, and voice-control will continue to be refined, and eventually make the “leap” to widespread consumer acceptance.

It’s a process Bacher believes benefits everyone; with the potential to open up avenues of technological interaction considered the realm of science fiction only a decade ago.

Jessica Feldman is a neuroscientist and PHD student at Brown University who is working with, among other projects, the BrainGate Group. BrainGate is a team of neurologists, neuroscientists, and other researchers all working towards a single goal: developing technologies that will restore the communication, mobility, and independence of people with neurological disease, injury, or limb loss.

The common term for this type of emerging technology is “brain-computer interfaces” (BCI), and it has the potential to change how we all interact with technology and the world around us at the most basic levels. Research into BCI began in the 1970s at UCLA under a grant from the National Science Foundation, and later DARPA. While research was initially limited to animals, the first human neuroprosthetic devices were implanted in the mid 1990s.

One of the largest barriers to effective BCI is translating neural activity into an effective machine signal, e.g., “telling” a robot arm to move in response to neural activity. Advances in methods of recording neural activity such as EEG and ECOG have improved this technology somewhat, but it’s still a very unnatural process. The goal of BrainGate and researchers like Feldman is to create natural “biomimetic” interfaces. Basically, thinking about picking up a glass of water would cause the machine to complete the complex collection of actions to pick up a glass of water.

The key to unlocking this type of interface lies with intracortical electrodes, says Feldman. These tiny wires are implanted in the brain’s cortex, and allow a single neuron’s activity to be recorded and mapped. Using a group of these wires, and taking up less than the space of a penny, allows researchers to record more complex signals.

This advance in recording neural activity opens up new avenues for helping individuals who are paralyzed or have lost a limb. Essentially, the signal of thinking about moving a lost limb can be sent to a prosthetic, causing the prosthetic limb to move. It’s an incredibly complex process, requiring specific hardware, processors, and technology. But the core necessity remains decoding the natural brain activity involved in any and every action. Says Feldman, “There’s so much basic science that goes on, even the smallest of neurophysiological questions are things that we’re looking at every day.”

However, the potential rewards are worth the effort, and “huge strides are being made,” says Feldman. The BrainGate neural device (currently in FDA human trials) has assisted individuals suffering from spinal cord injury, brainstem stroke, and ALS. At this stage, it allows individuals to control a computer cursor by thinking about the movement of their own paralyzed hand, or to use a mechanical arm to pick up and sip from a coffee cup simply by thinking about doing it. Since the technology involves physically implanting wires into the human brain, issues of safety and the longevity of the device must be thoroughly addressed. This is the BrainGate team’s next goal so that the device can move beyond the research stages.

The ultimate goal, says Feldman, “is to have a product that is an assistive medical device that can be used by any person with paralysis,” that can “restore their mobility and independence.” While the components are bulky now, as the technology continues to develop, miniaturization becomes more and more feasible. The BrainGate team currently is working on creating an autonomous system that is capable of wireless transfer. Feldman says that the aim is to create a device that can wirelessly communicate with a control unit that is small enough to be carried in a person’s pocket.

The current research and technology is focused on restoring mobility and independence to paralyzed individuals, but the potential applications are virtually unlimited. Decoding the signals of the brain and analyzing brain patterns opens up avenues for epilepsy research, artificial limbs, and understanding neurological diseases. Says Feldman, “at a basic neuroscience level, we’re learning about brain circuitry, neural circuitry, how things work. We’re able to study how the brain works for the first time at the micro level.” The real discovery in BrainGate’s research is how cognition becomes behavior.

In the end, the question isn’t what the implications of this research could be, but is there any area of human life that this technology would not effect?

Camilleri and his team have been studying brain responses for ten years. Now they have found one that is optimal for controlling a music player using eye movements. The system was originally developed to improve the quality of life of individuals with severely impaired motor abilities such as those with motor neuron disease or cerebral palsy.

The technology works by reading two key features of the user’s nervous system: the nerves that trigger muscular movement in the eyes and the way that the brain processes vision. The user can control the music player simply by looking at a series of flickering boxes on a computer screen. Each of the lights flicker at a certain frequency and as the user looks at them their brain synchronizes at the same rate. This brain pattern reading system, developed by Rosanne Zerafa relies on a system involving Steady State Visually Evoked potentials (SSVEPs).

Electrical signals sent by the brain are then picked up by a series of electrodes placed at specific locations on the user’s scalp. This process, known as electroencephalography (EEG), records the brain responses and converts the brain activity into a series of computer commands.

As the user looks at the boxes on the screen, the computer program is able to figure out the commands, allowing the music player to be controlled without the need of any physical movement. In order to adjust the volume, or change the song, the user just has to look at the corresponding box. The command takes effect in just seconds.

For people who have become paralyzed due to a spinal injury, the normal flow of brain signals through the spine and to muscles is disrupted. However, the cranial nerves are separate and link directly from the brain to certain areas of the body, bypassing the spine altogether. This particular brain-computer interface exploits one of these; the occulomotor nerve, which is responsible for the eye’s movements. This means that even an individual with complete body paralysis can still move their eyes over images on a screen.

This cutting age brain-computer interface system could lead the way for the development of similar user interfaces for tablets and smart phones. The concept could also be designed to aid with assisted living applications, for example.

The BCI system was presented at the 6th International IEEE/EMBS Neural Engineering Conference in San Diego, California by team member Dr. Owen Falzon.