Brain-Machine Interface (BMI) is promising to help disabled people restore their motor functions by decoding neural signals to control the neuro-prosthesis accomplishing their movement intention. However, when the subject purely uses its brain to control the devices (Brain Control, BC) instead of its real limb (Manual Control, MC), the neural signal will become different. The difference is even larger when the subject faces a new task that has not been trained before. The control mode switch and multi-task handling are crucial for clinical use in BMI, but existing decoding algorithms do not satisfy these requirements very well. In this thesis, we propose reinforcement learning-based decoding algorithms that discover new knowledge efficiently and memorize the old knowledge. And we evalua...[ Read more ]

Brain-Machine Interface (BMI) is promising to help disabled people restore their motor functions by decoding neural signals to control the neuro-prosthesis accomplishing their movement intention. However, when the subject purely uses its brain to control the devices (Brain Control, BC) instead of its real limb (Manual Control, MC), the neural signal will become different. The difference is even larger when the subject faces a new task that has not been trained before. The control mode switch and multi-task handling are crucial for clinical use in BMI, but existing decoding algorithms do not satisfy these requirements very well. In this thesis, we propose reinforcement learning-based decoding algorithms that discover new knowledge efficiently and memorize the old knowledge. And we evaluate our algorithms on three typical scenarios in BMI.

First, we propose a clustering-based kernel reinforcement learning algorithm and apply it from MC to BC. Previous studies have shown that neural patterns in BC are less distinguishable than in MC. To tackle this problem, we cluster the neural activities and only use the nearest cluster to decode. We test the algorithm on a monkey controlling a robotic platform. The experimental results show that our algorithm could achieve a better performance in BC using the local structure.

Second, based on the first algorithm, we propose a cluster alignment-based transfer learning method and apply it from one MC task to a new but related MC task. Researchers have found that neural activities might drift from one task to another, but there is shared information between two tasks. We treat each cluster as a knowledge basis and align the clusters between two tasks. We test the algorithm on both rats and a monkey. The results demonstrate that our algorithm could efficiently learn the new task and maintain the memory of the old one.

Finally, we combine the clustering-based kernel reinforcement learning algorithm with a linear state transition model and apply it to a continuous BC task. One major challenge is that the neural activities are non-stationary over time, and the decoder with fixed parameters cannot follow the changes during the continuous control process. We propose to combine a continuous state transition function with our kernel RL algorithm so that the reward signal could be used to adjust the parameters in an online fashion. We test the algorithm on rats performing the one-lever pressing BC task. Compared with the fixed Kalman Filter, our algorithm has a faster response speed and a larger number of successful trials, which demonstrates that our algorithm could follow the neural adaptation efficiently during the BC task.