Q+A: NeuroCONCISE ready for real-time BCI challenge

The NeuroCONCISE team from the University of Ulster have entered the BCI discipline at Cybathlon 2016. We caught up with team leader Damien Coyle and asked him the questions you all wanted to know.

How did you hear about Cybathlon?

I got the first flyer at a BCI workshop I went to, so it was fairly early on that I knew it was happening. From there it was direct contact, because I’m really integrated within the BCI community.

What do you think about competitions for technology development?

I’m always interested in competitions and have been working in BCI since 2002. These types of things are challenging, and they can be difficult or time-consuming but they’re really good to get involved in as they help you validate what you’re doing. It’s a key thing to test your systems and your technologies.

Does Cybathlon differ from other competitions you’ve entered in the challenges it poses?

It is totally different because this is a real-time application challenge, and you’re working with people with physical impairments, as they are the pilots. Other BCI competitions give you a set of pre-loaded data that has been recorded and you have to give the labels of the different activities that the person was doing. There will be data with test labels and without, so it’s all offline data analysis. There’s no real-time element to it at all.

Why do you think Cybathlon has such a good format?

The other competitions have been a good test of algorithms and training, but it wasn’t testing the real-world. Cybathlon will test technology in a real-world setting with real-time control, and all the challenges that brings. There are so many things that could go wrong, so you have to deal with all that and try to produce the best outcomes at the competition.

Damien Coyle

Damien Coyle

Do you have a pilot?

I’ve got a guy locally to here and he’s young and enthusiastic. I think he is going to be our pilot now.

Will you go to the rehearsal?

Yes we have plans to go to that. We secured some funding and that is our intention. But because we had issues finding a pilot we’ve had a little bit of a setback in making progress planning for that. I have secured about £6,000 to cover it, and we are definitely confident we can make it even though we haven’t had a lot of sessions with the pilot.

How is the training going?

We will ramp it up in June, because I haven’t had much time to work on the real-time element.  We have the game from Cybathlon and have been doing some testing with able-bodied participants. We are making slow but steady progress on it.

Is the game different to what you expected?

It’s a challenging enough game! You are kind of expected to have three classes* and a no control state. Whereas we’ve been used to having two classes with the no control state not really necessary.  With the Cybathlon game there are three different things you have to do: one of them is jump, one is speed, and one of them is kick your opponent. You have to do these at certain times, but on top of that if the system detects you are doing these outside the movements it slows you or your character falls. You have to have the no-control state where the system detects you are not trying to do one of the other three. That is a challenge.

[*Class = technology normally involves the imagination of left and right movements, which are known as classes. So you imagine left to move the cursor or character left, and vice versa. There are targets on the left or the right, and that dictates what class you are aiming for. So it is class left, class right, or class 1, class 2.]


NeuroCONCISE logo

Image: Courtesy of NeuroCONCISE


REVEALED: Which event are Cybathlon followers most interested in?

  • Brain Computer Interface voted the most interesting event
  • Followed by bike race and powered wheelchair race
  • Revolutionary BCI technology allows people to communicate without speaking or moving

The Brain Computer Interface (BCI) event has been voted the most interesting event by InsideCybathlon readers, pipping the bike and powered wheelchair races to first place.

A survey conduced by InsideCybathlon asked readers one simple question: “Which Cybathlon event are you most interested in?” and provided the options of: BCI, bike race, leg prosthetics race, powered exoskeleton race and powered wheelchair race.

Almost half of the respondents voted for BCI, while over 20 per cent opted for the bike race, which is designed for pilots with complete spinal cord injuries and allows them to cycle around a track via the use of Functional Electrical Stimulation devices which enables them to pedal.

The results of the survey can be seen below:

which cybathlon event are you most interested in

The Brain Computer Interface event will see pilots negotiate a series of obstacles as part of a racing game played on computers, using only their minds.

Pilots will have to send appropriate commands to move out of the way of oncoming objects, while four pilots can compete in one race at any time.

The game is the next strand of development of brain computer interface technology, which – aside from Cybathlon events – is designed to be used to allow disabled people to communicate.

The technology remains relatively unheard of, though, and teams such as the Brainstormers, from Essex, are hoping the publicity generated by Cybathlon will help shine more light on the work they and the other teams are doing.

The InsideCybathlon team visited the Brainstormers, where team leader Ana Matran-Fernandez explained how pilots will control the virtual cars in the race.

“We will ask them [the pilot] to think and move the car,” she said. “So they think of moving their left hand or foot, or right hand or foot and the car will move in that direction.”

You can see our video interview with Ana here:


BCI pilot told he’s ineligible

Jonathan Brough was one of the first people to sign up for Cybathlon. Now, two months on, he’s been told he can’t compete.

The reason? He has a pacemaker.

Given how excited he was about competing, he’s understandably gutted about the news, but ensures us he is talking to his GP to work out a way for him to be eligible again. At present, this leaves Team Imperial, involving scientists at Imperial College, pilot-less.

“Due to the fact that I have a pacemaker the organisers are saying, for safety reasons, I can no longer take part,” he told us.

“I’m still working with Imperial College in designing and testing and intend to go to Cybathlon 2016, its just that at this point I will not be allowed to take part in the races. I have discussed this with my G.P. who is currently looking into it.”

Linda Seward, of NCCR Robotics, explained the ruling to us:

“The reason for the exclusion criteria for pacemakers is that as the participating devices may not be CE certified they may interfere with a pacemaker, so it’s a safety precaution.”

If you didn’t read our original interview with Jonathan, he contracted meningitis in 2007 and has been paralysed ever since.  Now 26, he lives near Stroud with two round-the-clock carers, but did not let it stop him completing a degree at Plymouth University.

As part of his preparation for Cybathlon, he had just started learning how to use a BCI, as he normally speaks with air from his ventilator. To control his wheelchair he uses a lip joystick, but the Rio Tinto team had been developing eye control for him.

Fortunately, Jonathan has found something to take his mind off the disappointment.

“I’m still finding it difficult to understand their decision but am hoping to go skiing in Milton Keynes next week.”

For a former skiing enthusiast, that is certain to lift his mood.

Jonathan Brough is aided by his carers.

Team effort: Jonathan Brough is aided by his carers.

Images: Courtesy of Jonathan Brough


VIDEO: Brainstormers and BCIs: On your marks, set… think!

Deep in the bowels of Essex University, sitting far below the student hubbub of shops, bars and hangovers, some of the most intelligent researchers in the country are plotting.

Plotting new ways to improve the lives of thousands of disabled people. Plotting innovative technologies which have never been seen before. Plotting their assault on the 2016 Cybathlon championship.

The aptly named Brainstormers, a team of scientists at the School of Computer Science and Electronic Engineering, will be competing in next year’s event as an aside from their revolutionary studies into brain computer interfaces (BCIs).

Hidden away down a few damp flights of stairs and a long, thin corridor filled with identical doors and not much noise, it would be easy to forget that the Brainstormers are living on the front lines of one of the most exciting pieces of technology in the UK.

Song-Jae wears a cap used to capture brain activity

Feeling wired: Youngjae Song gets ready to test the BCI

BCIs are still developing, but they essentially allow the Brainstormers (and others across the world such as scientists at the Center for Integrative Neuroscience at Berkeley and the Life Science Center in Tokyo) to capture brain signals and turn them into commands.

In other words, they let you communicate without speaking or moving – meaning humans will be able to operate computers, robots or virtual reality environments using just neurosignals, even if they have no control of their muscles.

The Brainstormers team consists of five PhD students: team leader Ana Matran-Fernandez, Youngjae Song, Davide Valeriani, Amir Jahangiri, Dimitrios Andreou and Cambridge University’s Christian O’Connell. Working alongside them are Dr. Javier Asensio-Cubero and Dr. Francisco Sepulveda.

The Brainstormers team: (From left to right) Youngjae Song, Dr Francisco Sepulveda (lower row), Ana Matran-Fernandez, Davide Valeriani, Dr Javier Asenso-Cubero

Game-changers: (From left to right) Youngjae Song, Dr Francisco Sepulveda (lower row), Ana Matran-Fernandez, Davide Valeriani, Dr Javier Asenso-Cubero

Matran-Fernandez says the technology – if developed properly – can have a massive impact on those that need help communicating and says Cybathlon will be the perfect place to showcase the potential of these interfaces.

“We think Cybathlon is going to be really good for BCIs,” she said.”So when we read about Cybathlon we thought it would be a great way to get known by external people and try to help.

“It’s going to expose the field so more people will know about it and that’s great because they have the potential to do a lot of good.”

BCIs won’t just be shaping the field for disabled people, though. As Sepulveda points out, they could also have a huge impact on the gaming industry as companies and competitors look for new, imaginative ways to test themselves.

The disparity between ideal brainwaves and typical signals shows how difficult the task is

Nature of the beast: The disparity between ideal brainwaves and typical signals shows how difficult the task is

With the support of BioSemi, one of the nation’s leading electro-physiologic equipment suppliers, the team are currently on the hunt for a pilot to compete as part of the Brainstormers team.

“Between now and Cybathlon, we will first be looking for more sponsors and we’ll also be looking for a pilot, which is the main difficulty at this point,” Matran-Fernandez says.

“Other than that, we’re going to be training hard for the mock test and hopefully we will have someone to help do that. Once we do, we’ll know how things are and we’ll be able to be more specific with our preparation.”

But how does it work? We were shown into a small windowless room in a corner where the machine was kept as Youngjae Song put the equipment on – a lovely piece of gear which looked like a washed-out red swimming cap, except it was crawling with worm-like wires, which pick up the neurosignals.

“When you’re using the interface, you display something on the screen and you can time the brain signals based on what is being played and interpret them one way or the other,” Matran-Fernandez explains.

Although their game involves either a car or a horse, which pilots have to turn round obstacles and past corners, that may change in the competition itself.

“We don’t really know what Cybathlon is going to be like but what we’re expecting is some sort of track and racing video game which the pilot sees on his screen.  What we think is going to happen is that the person has to drive the car to the left or the right,” she says as we watch the game in motion.

“We will ask them to think and move the car, so they think of moving their left hand or foot, or right hand or foot and the car will move in that direction. That’s one of the several types of games there are.”

Song-Jae plays a BCI game designed by the Brainstormers team

Head in the game: Youngjae Song locks into the BCI

Of course, it’s not completely flawless technology at the moment, as Dr. Asensio-Cubero points out.

“It’s not really accurate or reliable – you may get 60 per cent accuracy or maybe 80 per cent if the subject is really good,” he says. “This event requires a really high standard of pilot.”

Another way of working with BCIs is to examine “P300” signals – brainwaves which appear soon after the presentation of a stimulus of interest, according to Matran-Fernandez.

“So if you’re shown what we call ‘distractors’, and you see the thing you’re looking for which is of interest, your brain produces these signals and the computer will know what you’re interested in.

“Based on that, there are games called ‘Spellers’. With these, there are lots of rows of letters or numbers, and when a row or column which contains the letter you want flashes we can see it on the screen.”

Sound simple? Maybe. Does it look simple? Not at all. These rows and numbers flashed at such speed that it was exhausting just looking at it, let alone trying to spell a word by focusing on individual symbols.

But with a skilled and experienced pilot, it’s clear to see how this technology could change the lives of people who are ‘locked-in’ and cannot communicate in other way.

“It’s for people who cannot even move their eyes,” says Asensio-Cubero. Nothing hits home how important their work is better than that sentence.

BCIs are perhaps the most iconic event in Cybathlon – nothing else pushes the boundaries of technology quite so far. Although crouching in the basements of Colchester with barely any natural light certainly adds to the experience, it’s clear to see that this is a remarkable concept using remarkable technology. And, frankly, as we watched Song twist and turn a virtual car on a virtual track using just his thoughts, it’s fair to say that it’s remarkably cool.

If you think you would be a suitable pilot for the Brainstormers, get in touch with them here:

And listen to the extended chat with the team:

Image: Sam Dean