Using a BCI the robot was able to find targets that the person could see but the robot could not (Photo: Iturrate et al., 2015).

Brain-Machine Interfaces (BMIs) — where brain waves captured by electrodes on the skin are used to control external devices such as a robotic prosthetic — are a promising tool for helping people who have lost motor control due to injury or illness. However, learning to operate a BMI can be very time consuming. In a paper published in Nature Scientific Reports, a group from CNBI, EPFL and NCCR Robotics show how their new feedback system can speed up the training process by detecting error messages from the brain and adapting accordingly.

One issue that bars the use of BMIs in everyday life for those with disabilities is the amount of time required to train users, who must learn to modulate their thought processes before their brain signals are clear enough to control an external machine. For example, to move a robotic prosthetic arm, a person must actively think about moving their arm — a thought process that uses significantly more brainpower than the subconscious thought required to move a natural arm. Furthermore, even with extensive training, users are often not able to perform complex movements.

It has been observed, however, that the brain emits very different waves when it experiences success at controlling a BMI than when it experiences failure. With this in mind, the research team developed a new feedback system that records error signals from the brain (called ‘error-related potentials’, or ErrPs) and uses these to evaluate whether or not the correct movement has been achieved. The system then adapts the movement until it finds the correct one, becoming more accurate the longer it is in use.

In order to determine the ErrP, twelve subjects were asked to watch a machine perform 350 separate movements, where the machine was programmed to make the wrong movement in 20% of cases. This step took an average of 25 minutes. After this first training stage, each subject performed three experiments where they attempted to locate a specific target using the robotic arm. As expected, the time taken to locate a target reduced as the experiment continued.

Three experiments showed that a robot improved its ability to find the position of a fixed point using error-related brain activity. (Iturrate et al. 2015)

https://youtu.be/jAtcVlTqxeA

This new approach finds obvious applicability in the field of neuroprosthesis, particularly for those with degenerative neurological conditions who find that their requirements change over time. The system also has the potential to automatically adapt itself without the need for retraining or reprogramming.

Reference

I. Iturrate, R. Chavarriaga, L. Montesano, J.  Minguez and J. del R. Millán, “Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control,” Nature Scientific Reports, vol. 5, Article number: 13893, 2015. doi:10.1038/srep13893

If you liked this article, you may also be interested in:

• DARPA’s HAPTIX project hopes to provide prosthetic hands with sense of touch
• Lifehand 2 prosthetic grips and senses like a real hand
• Control strategies for active lower extremity prosthetics and orthotics
• The gateway to advanced neuroprosthetics: Jessica Feldman talks BrainGate and BCI
• Bionic athletes compete in disciplines drawn from everyday life

See all the latest robotics news on Robohub, or sign up for our weekly newsletter.

Researchers from the Keio Institue of Pure and Applied Sciences (KiPAS) are working to reveal the mechanism by which newly acquired knowledge and information is transmitted and evolves among organisms with intelligence. This research is being done through pseudo-augmentation of body structures using a brain-machine interface, to investigate the process by which groups of individuals adapt to novel bodies and environments, using methods from the natural sciences.

Junichi Ushiba and Giulia Cisotto explain their work:

“Originally, I was very interested in the fact that we ourselves evolved from monkeys. The form of a body changes gradually through the process of evolution – for example, having a tail or not having a tail – and species that survive are using that specially evolved body effectively. Thinking about what this implies, even if a body changes, unless the brain that controls the body has the ability to adapt to the change, then naturally, an organism can’t utilize the adaptation effectively like the rest of its body. So, I think that one important keyword Concept in evolution is that the brain itself actually has the ability to adapt flexibly to external objects even more than we think it does. That’s precisely why I think there may be a process where a part of the body that has suddenly changed is incorporated, as if it was a part of the organism’s body, and the organism adapts and evolves accordingly. I wondered if this kind of process couldn’t actually be verified using methods from the natural sciences, and that was the motivation for this research.”

The method used in this research is to observe, in the laboratory, a group of about ten people who have an artificial tail, which can drive a brain-machine interface, attached to their body. Participants learn how to move the artificial tail, and attempt to use it skillfully, like their own body, through trial and error. They also share the ways in which they’ve learned to use the brain-machine interface with other people.

“In the beginning the person doesn’t know how to use the tail, but with some sort of training it’s possible for the person to modulate his or her brain activity in a way to control the tail.

The second step will be to study, using the same technology, some sort of heredity, so some sort of transfer of these new acquired skills to other people. So in the second step we will have more than one person in the same room and the first person, who has already acquired the skill to correctly move the tail, by moving around the room and talking with the other people, can transfer these acquired skills to the other people. So that in a certain period of time, all the other people gradually acquire the same skills, probably in another way, but with this technology we can monitor how each person, individually, can acquire the new skill.”

In physical training methods for sports, dance, and music, and in traditional arts and crafts, there are a variety of long-established approaches and styles, so universal principles for passing on such techniques have not been established. By contrast, the results of this research will provide scientific principles regarding control laws, learning processes, and ways of transmitting knowledge to other people. So, this research could contribute to scientific advances in industrial and cultural activities.

“Unless there are mechanisms that enable information that’s been correctly acquired by the mind to be transmitted to other people reliably, cultural activities that get sustained for hundreds of years, or go on to evolve further, couldn’t exist. Through this kind of research using tails, what I really want to understand is the process by which intangible information is created in the mind and transmitted to other people, so we can reveal some aspects of how culture and civilization took shape, using methods from the natural sciences.”

Brain-computer interfacing (BCI) is a rapidly growing field that offers huge potential for many applications, such as medical grade BCIs to help people with sensory-motor disabilities. Currently, a number of researchers are developing more affordable BCI systems designed to address a wider range of neurotherapeutic applications.

Conor Russomanno is the founder and research developer of OpenBCI, a low-cost open-source hardware platform that records the brain’s electrical signals and uses devices and software languages to make the data easily accessible. Russomanno and co-founder Joel Murphy aim to accelerate the advancement of BCI through collaborative hardware and software development.

The OpenBCI project began earlier this year after raising over $200K through a Kickstarter campaign (more than doubling their initial goal). Even in the early stages, OpenBCI is already drawing a lot of attention and is having interesting repercussions in the BCI world.

“There’s a lot of really exciting stuff that’s going to happen in the near future and in the not so distant future, but I think the one thing that we’re hoping to achieve with open BCI is to really lower the barrier of entry both in terms of educational materials and also cost,” says Russomanno.

http://youtu.be/a55_EfFtysc

A number of consumer-grade EEG devices that have a relatively low channel capacity are available, but higher quality, medical and research grade EEG systems are much more expensive than a standard OpenBCI kit, which makes them less accessible. One of the advantages of Russomanno’s and Murphy’s development is that whereas previously, one medical grade system was used in a classroom or research laboratory, now, the same amount of money can be used to acquire  enough OpenBCI systems to provide every student with their own device.

“This is reflected in our consumer base. A huge proportion of our customers are students, graduate-level researchers and professors who want to use OpenBCI as a learning tool, explains Russomanno.

Another of OpenBCI’s exciting potentials is the fact that it is fully open source. Russomanno and Murphy are using a lot of their resources to create an infrastructure in which people are not just able to use the technology but can also share their experiences and connect with other people using the technology. This enables users to share tutorials, getting started guides and also their own individual projects. In this sense, the creators have not only provided a tool, they are also establishing a level of enthusiasm for an unprecedented unicentric development.

The fact that such enthusiasm is being drummed up in many sectors from academia to the business world, marks a big step for BCI. How far into the mainstream does OpenBCI have the potential to go?

“I think that to be realistic about the technology it’s important to realize that, especially in the commercial space, technology like this has a tendency to get mislabelled and misrepresented by media sources. When you see the catchphrase ‘mind controlled robot’ you think of someone sitting in a chair with electrodes plugged into their head having ultimate control over many, many dimensions of a robot’s functionality.” Says Russomanno.

In reality, however, in the current state, mind controlled robots, at least using OpenBCI, translates to looking at a flashing screen and having the frequency of the flashing induce a similar frequency in the brain, and then telling the robot to turn left.

“We’re are many steps away from wearing a device that knows what turning your car on, versus opening your door look like. So it’s important to be realistic about what the technology is capable of,” Russomanno explains.

Russomanno points out that for BCI to advance, it’s essential for people to work together. Although there is some competition within this space, he emphasizes the importance of cooperation. Progress from any one company is going to help the industry as a whole. People in tested in joining the OpenBCI community can buy the technology, make use of the many available resources and share their experiences and this will work toward pushing BCI forward into the future.

Image credit: 4bpblogspot.com

Link to audio file (23:28), or  listen on iTunes

In this interview, TechEmergence talks to Dr. Beata Jarosiewicz from Brown’s BrainGate project about the science of making sense of the brain’s inner electrical activity, and how the brain might be used to control a variety of devices if calibrated correctly. 

Researchers at BrainGate have given paralyzed patients the ability to move robotic arms and check emails with their thoughts alone – thanks to experimental neural implants in their motor cortex. Dr. Beata Jarosiewicz has been with BrainGate project for years, with a specific focus on interpreting neurological signals and turning them into actionable outputs.

The BrainGate Group is on the cutting edge of an emerging technology – brain-machine interfaces. Their advances have allowed individuals with extensive paralysis and locked-in syndrome to interact with the world around them by controlling a computer mouse with their mind. Moving a mouse is something that most of us take for granted, but an individual with locked-in syndrome, that is fully awake and conscious, has no other means of producing speech, move limbs, or make facial movements.

Through a device implanted in the skull, the BrainGate device interfaces with a computer, allowing the individual to interact with their environment. This technology is made possible in part by extensive mapping of the neural activity of the brain, but revolutionary hardware designed by engineers like Bacher have also been key.

Bacher began working on the BrainGate technology by conducting brain-machine interface research on non-human primates. Or, as Bacher describes it, “basically having monkeys playing video games with their brains.” This research helped Bacher and other researchers understand how the brain works and begin development of the assistive technologies. When the opportunity to join the BrainGate group presented itself, Bacher jumped at the chance.

Says Bacher, “Although I’m interested in the basic neuroscience and love the pure technology aspects of it, thinking about how this translates to improve the quality of life for people in the real world has always been the underlying motivation for me.”

Outside of his work on BrainGate, Bacher is passionate about connecting disabled individuals with assistive technologies. This passion led Bacher to found the Speak Your Mind Foundation, which creates, distributes, and supports assistive communication technologies for individuals who are unable to effectively communicate due to neurological injury and disease. Says Bacher, “there is still a huge gape between technology creators and healthcare providers.” Bacher hopes to help bridge this gap and bring these technologies to the healthcare market.

Bacher’s approach also takes worldwide emerging technology trends in hands-free computing, robotics, and other areas, and finds ways to translate them into helping disabled individuals. Using technologies developed for larger markets helps lower the costs in developing applications for assistive technologies.  While the technological solutions are currently tailored for specific individuals involved in clinical trials, the solutions developed for them have potential applications for larger groups.

Says Bacher, “if we develop a piece of technology for someone with a spinal cord injury who is moving their head to control a computer, there’s no reason that can’t help someone whose suffered a stroke.”

Human-computer interface technology is still largely limited to touch, keyboard, and mouse. The research conducted by Bacher and those like him has the potential to change this, and as these technologies progress, their potential applications widen.

Says Bacher, “clearly, a mouse and a keyboard, even today seems like an antiquated way to interact with technology. We’ve seen developments like touch-screen technology and voice-control come along way, but it’s still not mainstream. In order to really penetrate the commercial marketplace, there’s a high threshold for how well something has to work.”

While current research is focused on assistive technologies for disabled individuals, the possible applications don’t end there. Part of the reason for this is the “symbiotic” relationship developing between consumer and assistive technologies. Smart phones can provide the “brains” for assistive devices. Eye motion-capture technologies developed for paralyzed individuals open up new ways for the general public to interact with technology, and hands free technologies benefit individuals without the use of their arms. Bacher believes that technologies like Google Glass, Leap Motion, Kinect, and voice-control will continue to be refined, and eventually make the “leap” to widespread consumer acceptance.

It’s a process Bacher believes benefits everyone; with the potential to open up avenues of technological interaction considered the realm of science fiction only a decade ago.

Jessica Feldman is a neuroscientist and PHD student at Brown University who is working with, among other projects, the BrainGate Group. BrainGate is a team of neurologists, neuroscientists, and other researchers all working towards a single goal: developing technologies that will restore the communication, mobility, and independence of people with neurological disease, injury, or limb loss.

The common term for this type of emerging technology is “brain-computer interfaces” (BCI), and it has the potential to change how we all interact with technology and the world around us at the most basic levels. Research into BCI began in the 1970s at UCLA under a grant from the National Science Foundation, and later DARPA. While research was initially limited to animals, the first human neuroprosthetic devices were implanted in the mid 1990s.

One of the largest barriers to effective BCI is translating neural activity into an effective machine signal, e.g., “telling” a robot arm to move in response to neural activity. Advances in methods of recording neural activity such as EEG and ECOG have improved this technology somewhat, but it’s still a very unnatural process. The goal of BrainGate and researchers like Feldman is to create natural “biomimetic” interfaces. Basically, thinking about picking up a glass of water would cause the machine to complete the complex collection of actions to pick up a glass of water.

The key to unlocking this type of interface lies with intracortical electrodes, says Feldman. These tiny wires are implanted in the brain’s cortex, and allow a single neuron’s activity to be recorded and mapped. Using a group of these wires, and taking up less than the space of a penny, allows researchers to record more complex signals.

This advance in recording neural activity opens up new avenues for helping individuals who are paralyzed or have lost a limb. Essentially, the signal of thinking about moving a lost limb can be sent to a prosthetic, causing the prosthetic limb to move. It’s an incredibly complex process, requiring specific hardware, processors, and technology. But the core necessity remains decoding the natural brain activity involved in any and every action. Says Feldman, “There’s so much basic science that goes on, even the smallest of neurophysiological questions are things that we’re looking at every day.”

However, the potential rewards are worth the effort, and “huge strides are being made,” says Feldman. The BrainGate neural device (currently in FDA human trials) has assisted individuals suffering from spinal cord injury, brainstem stroke, and ALS. At this stage, it allows individuals to control a computer cursor by thinking about the movement of their own paralyzed hand, or to use a mechanical arm to pick up and sip from a coffee cup simply by thinking about doing it. Since the technology involves physically implanting wires into the human brain, issues of safety and the longevity of the device must be thoroughly addressed. This is the BrainGate team’s next goal so that the device can move beyond the research stages.

The ultimate goal, says Feldman, “is to have a product that is an assistive medical device that can be used by any person with paralysis,” that can “restore their mobility and independence.” While the components are bulky now, as the technology continues to develop, miniaturization becomes more and more feasible. The BrainGate team currently is working on creating an autonomous system that is capable of wireless transfer. Feldman says that the aim is to create a device that can wirelessly communicate with a control unit that is small enough to be carried in a person’s pocket.

The current research and technology is focused on restoring mobility and independence to paralyzed individuals, but the potential applications are virtually unlimited. Decoding the signals of the brain and analyzing brain patterns opens up avenues for epilepsy research, artificial limbs, and understanding neurological diseases. Says Feldman, “at a basic neuroscience level, we’re learning about brain circuitry, neural circuitry, how things work. We’re able to study how the brain works for the first time at the micro level.” The real discovery in BrainGate’s research is how cognition becomes behavior.

In the end, the question isn’t what the implications of this research could be, but is there any area of human life that this technology would not effect?

Camilleri and his team have been studying brain responses for ten years. Now they have found one that is optimal for controlling a music player using eye movements. The system was originally developed to improve the quality of life of individuals with severely impaired motor abilities such as those with motor neuron disease or cerebral palsy.

The technology works by reading two key features of the user’s nervous system: the nerves that trigger muscular movement in the eyes and the way that the brain processes vision. The user can control the music player simply by looking at a series of flickering boxes on a computer screen. Each of the lights flicker at a certain frequency and as the user looks at them their brain synchronizes at the same rate. This brain pattern reading system, developed by Rosanne Zerafa relies on a system involving Steady State Visually Evoked potentials (SSVEPs).

Electrical signals sent by the brain are then picked up by a series of electrodes placed at specific locations on the user’s scalp. This process, known as electroencephalography (EEG), records the brain responses and converts the brain activity into a series of computer commands.

As the user looks at the boxes on the screen, the computer program is able to figure out the commands, allowing the music player to be controlled without the need of any physical movement. In order to adjust the volume, or change the song, the user just has to look at the corresponding box. The command takes effect in just seconds.

For people who have become paralyzed due to a spinal injury, the normal flow of brain signals through the spine and to muscles is disrupted. However, the cranial nerves are separate and link directly from the brain to certain areas of the body, bypassing the spine altogether. This particular brain-computer interface exploits one of these; the occulomotor nerve, which is responsible for the eye’s movements. This means that even an individual with complete body paralysis can still move their eyes over images on a screen.

This cutting age brain-computer interface system could lead the way for the development of similar user interfaces for tablets and smart phones. The concept could also be designed to aid with assisted living applications, for example.

The BCI system was presented at the 6th International IEEE/EMBS Neural Engineering Conference in San Diego, California by team member Dr. Owen Falzon.