Neural Prosthetic Control: Cognitive Factors & Adaptation

[00:00:08]all right thank you all for uh coming we'd like to start the ri seminar series for for fall of 2012 and we're very honored to have with us as our inaugural speaker professor stephen chase he has a joint appointment at the cnbc which is the center for the neural basis of cognition and the biomedical engineering department at cmu he did his phd at johns hopkins and joined as a postdoc at cmu in 2006 and he's been an assistant professor since 2011. he does a lot of interesting work and i'm not going to attempt to you know coalesce into a story but uh it's it's largely to do with coding uh the the end flow of information in the ring right lots of interesting work and we look forward to the talk thank you very much i'm excited to be here and uh excited to kick off the series so as the title implies i'm going to talk a lot about neural prosthetics and the idea behind neural prosthetics is is pretty simple say we have a patient whose spinal cord injured or suffers from some other motor disorder where the brain is entirely intact but for whatever reason they can't communicate signals uh to the muscles and actually interact with the external world and what we would like to do is design a device that can read out motor intention signals or some kind of intention signal from the brain and then use that signal to directly actuate either a cursor on a computer screen or a robotic arm something like that and restore the ability for these patients to interact with the external world and so a typical schematic of a neural prosthetic system looks something like this you have a patient that patient generates some kind of motor intention signal that intention signal flows down into primary motor cortex and drives the activity of neurons in primary motor cortex and so what we do is we record from those neurons using chronically implanted microelectrode arrays these are typically they're called utah arrays there's 100 channels on them and we can record from roughly about 100 neurons in primary motor cortex and if we know how those neurons represent motor intent we can then design a decoder to infer what that motor intent is and directly use that to drive say a cursor on a computer screen or a robotic arm and then the subject sees that resultant movement and closes the loop generates a new intense signal to try and drive that device to to perform the action he's he's trying to do so really the only difference between these kinds of devices and the kinds of devices that you and the robotics community are used to working with is the fact that we have actually a live subject in the loop and this is a blessing and a curse for us it's a it's a blessing because it means that we can spend a lot less time dividing uh designing things like intelligent planning algorithms because we have that in the loop already it's a curse because unfortunately we don't really understand how that intelligent planning algorithm is working and so a lot of times our decoding approaches aren't actually appropriate for what the subject is doing so for those who haven't seen it this is what one of these devices actually looks like in action this is a video released by the schwartz lab back in 2008 and they have a monkey implanted with one of these utah electrode recording arrays it's recording about 100 neurons in primary motor cortex and they're using those signals to drive this robotic arm and the monkey is controlling both the 3d endpoint position of the robotic arm as well as the 1d gripper aperture of the arm and when you have a highly motivated subject as this monkey is because he really loves marshmallows then the subject can learn to use his device to do something useful like feed himself marshmallows so what you may not know is that is is how that device worked and the basis of control behind that device was something called the population vector algorithm so i'll explain what that does so the basis of all prosthetic control is is is if we're going to tap motor intent signals out of the motor cortex we need to understand how those neurons are representing intent and a lot of what we understand about how that works comes from a series of studies started by georgiopolis back in 1982 and what georgiopolis did is he had monkeys perform reaching movements using using their arm and they perform at uh what they call a center out reaching task where he'll have his arm in the center of a workspace and then he'll make movements to targets equally spaced on on a circle around the outside and georgiopolis recorded single neurons in primary motor cortex while monkeys performed this task and he just looked at at their firing rates and what he found is that when the movement was in certain regions of space that neuron would fire quite a lot and when the movement was in other regions of space down in the lower corner there the neuron would stop firing and in fact if you plot the firing rate as a function of the angle of movement actually a lot of neurons exhibit this nice cosine tuning curve and so we see now this is actually a relatively surprising finding because until this time people thought neurons in primary motor cortex really were there what their job was was to drive muscles directly and so you'd expect that the the firing rates of neurons would be more related to things like the force that the muscle was generating or to things like the torques that the joints were generating and these are relatively complex multi-joint movements but what georgeopoulos found is that we could actually describe the activity of the neurons using a relatively simple code just the direction of movement in extrinsic space which makes doing something like inferring direction of movement much much simpler so these cosine tuning curves can be parameterized by three numbers the first is denoted in red here it's the preferred direction of the neuron that's the direction of movement that this neuron is most responsive to the second in green is the modulation depth of the cell which describes its dynamic range of modulation and the third is its average firing rate what we call the baseline activity of the cell so the way so the first thing i should mention is that cosine tuning all it is is linear tuning all right to understand that we can write a linear equation on the board where if the firing rate of the cell is just some linear function of coefficients times the direction of movement dx and dy in 2d space and we can rearrange this equation and draw it out in vector form where we have a vector b of b coefficients and then we have the direction vector and if the direction vectors automatically a unit vector if we rescale that coefficient vector to be a unit vector by taking a scalar length component the modulation depth it turns out outside then you can see when we take the dot product of these two unit vectors what we get is the cosine of the angle between them and so cosine tuning is really your statement that a lot of neurons in the brain linearly encode the direction of movement and this is also nice for the design of algorithms because most of our algorithms assume linear encoding so the way the population vector algorithm works in a nutshell is that we record from these neurons and we sample how often they're firing spikes over some small window of time say 30 milliseconds so we count the number of spikes it fires over that 30 millisecond bin divide by the time that gives us a firing rate now we normalize those rates by subtracting off the baseline firing rate and dividing by the modulation depth of the cell so that gives us a normalized estimate of that neuron's firing rate which is very noisy because it's been taken over 30 milliseconds and so what we do is we smooth that estimate by taking a running average of it over the last five time bins last 150 milliseconds and that gives us a smoother estimate still relatively fast of that neuron's firing rate and then what we do is we use that smooth normalized firing rate to weight the length of a vector that points in that cell's preferred direction and the idea here is that the more this cell fires the longer that vector is going to be the longer that vector that points in its preferred direction is going to be and then as the name population vector average implies what we do is we take the average of all those little vectors we add them up in space and the resultant vector we scale it by some constant ks to turn it into units of velocity and that is the velocity of the thing we're going to actuate be it a robotic arm or be it a cursor on a computer screen and then we just add those vectors up tip to tail we integrate them to get the position of those devices so that is how the population vector algorithm works now there are a lot of good reasons why it should not actually be a good algorithm for driving a prosthetic device so the first of these is that i told you that a lot of neurons are cosine two and they have that linearly related to the direction of movement well it turns out that's not quite right and the reason it's not quite right is because neurons can't have negative firing rates and so a lot of time what they show is this rectification behavior where when the movements go in the opposite direction they fall off of that linear line so they're not quite linearly tuned and also i've told you that they tend to follow these cosine tuning curves it turns out when you measure that very very accurately like american and georgiopolis did back in 2000 what you find is that instead of having a half width of 90 degrees like you would expect from a cosine tuning curve a lot of times those half widths are actually about 60 degrees so they're narrower than cosine on average the third problem with the pva is it assumes that your tuning curves are stationary and it turns out the tunics can change for a lot of reasons and so this is an experiment that was done by by rockne at all back in 2007 and where they just had a monkey perform the center out reaching task for an hour and then they divided the session up into the first 20 minutes in the last 20 minutes and they measured tuning curves for those 20 minutes and they compared them and what they found is that they got significant changes in the tuning curves of most of the neurons they recorded so these tuning curves tend to drift around over time for various reasons and the final reason why the population vector algorithm shouldn't actually work as well as it does is because as a statistical estimate of direction it's actually terrible and it's terrible because it's a biased estimator and you can see that it's biased by just playing this little thought experiment so assume you have two neurons who are perfectly cosine tuned so you have one neuron the blue neuron here which has a preferred direction of zero degrees directly to the right and you have a second neuron that has a preferred direction of 45 degrees sort of up and to the right and let's say those are the only two nodes you're recording from and you're trying to drive a cursor on a computer screen now when the subject is trying to aim directly to the right he's going to activate that first neuron it's going to fire above its baseline firing rate because that's in that neuron's preferred direction he's also going to activate the second neuron because it's close to that neuron's preferred direction and so both of them are going to have positive contributions to the population vector average and the resultant movement is going to be not directly to the right but it's going to be a little bit up into the right and you can play this game assuming he's trying to aim at points all the way around that circle and what you get are that the movements that actually result tend to be clustered along the axis that describes the majority of tuning curves of those neurons so it's a biased algorithm so how is it possible like i showed you in the video the monkey was able to feed himself and it seemed like without too much trouble how is that possible and the reason that's possible is because the brain adapts and so what i showed you about that rectification behavior under hand control when the monkey's making these reaction movements yes it's true these neurons deviate from linearity but it turns out if you take the same neurons and you measure them in a context when the monkey's actually using the prosthetic device a lot of times those neurons look far more linear and in fact if you look across time while the monkey trains with that prosthetic device and you measure the tuning curves of those neurons when he's making natural reaching movements and then you take the same neurons and measure their tuning curves when he's using the device and you compute the r squares of those two linear fits into brain control the r squared of that linear fit increases over time and in fact if you take the difference between the brain controller squared and hand control r squared it shows a sharp increase over over the course of months and the result seems to be that the longer the subject uses a decoder that assumes the neuron should be linear the more linear those neurons become so now i mentioned to you that the population vector algorithm is a biased estimator now this was mentioned to valiste at all when they submitted their paper to nature and the reviews went something like this and i'm paraphrasing but something like it's a shame that you all use the population vector algorithm the population vector algorithm is provably mathematically inferior and it would have been nice to have seen how good the control would have been had you used a more reasonable decoding algorithm and it's true you can prove that the pva is biased and the way you can prove it yourself you can prove it mathematically using that thought experiment like we did or you can take neurons that you record while the monkey is making an actual reaching movement with his arm and then you can invert that problem and try and predict where that arm was at every instant in time and you can do that prediction using the population vector algorithm and you can do that prediction using the optimal linear estimator which is an unbiased version of the same thing and for this offline trajectory reconstruction case the ole the optimal linear estimator outperforms the population vector algorithm every time significantly but the question is how well do they compare online so we did an experiment where we had a monkey push a cursor around in 2d using a population vector algorithm and then after he'd done that for a while after he made 15 reaches to each of our eight targets we changed it on him without telling him in the middle it's between two reaches we changed the decoding algorithm on him to the optimal linear estimator we had him use it for 15 reaches to each of those targets and then we'd switch it back and that sort of thing and what i'm showing you here are the trajectories from every one of the reaches in that experiment his success rate by the way through this experiment was a hundred percent so changing the decoder on him in the middle didn't slow him down at all and just looking at these you'd be hard-pressed to tell me which was the bias decoder and which was the unbiased decoder so it could be that for this particular sample of neurons we had it didn't happen to have a lot of bias in it maybe the preferred directions were nicely uniformly distributed or something like that it turns out that's not the case to show that we can take the same firing rates when the monkey is using the population vector algorithm and we can decode them offline using the optimal learner estimator and what you get are the trajectories shown here which show those characteristic biased curves most of those trajectories are missing the targets in characteristic directions and you can do that for the same for the firing rates when the subject was using the optimal learner estimator if we decode those firing rates using the population vector algorithm we get that same bias in the opposite direction and so what it's looking like is that the subject is able to compensate for the bias in this decoder and he does it nearly instantly all right so the point here is that offline mean squared error of our reconstructed arm movements is equivalent to the bias squared plus variance you can break it down in those ways but it looks like online subjects can compensate for that bias and the point here is that offline estimates of how well an algorithm will perform do not necessarily translate into online performance gains or thinking about a little bit more broadly this severely changes how we ought to think about designing our decoding algorithms now usually the first thing we think as statisticians or as or as you know anybody who does mathematics the first thing you think about when you're trying to design an estimator is you're trying to to design an unbiased estimator it tends to be the first thing we do and what this results show us is that maybe we need to expand our thinking and and not immediately rule out the class of unbiased estimators the class of bias estimators that's actually out there okay so this talk is going to be broken down roughly into two parts so the first part i'm going to describe some of the research we've done to try and understand these learning processes that happen while a subject is is using a brain computer interface to see if we can sort of understand if there are limitations to those learning processes or perhaps understand how to design a decoder that might better take advantage of those learning processes and the second part of the talk is i'm going to talk about a new decoding approach that we use that appears to give improved cursor control so how do we investigate learning in our brain computer interfaces and and the way people uh as psychophysicists especially investigate learning in the lab is by using things like visual motor rotations and visual motor gains so visual motor rotation we're all relatively familiar with this when you go to grab your mouse and you don't grab it quite right and you push it straight forward say you grab it at an angle what happens is the cursor doesn't move straight straight up on the screen it moves off at an angle now most of us would just readjust our hand and do it but if you were to keep using the mouse at that odd angle you would learn the direction in which to push the mouse in order to make the cursor move in the direction you want it turns out humans are very very good at adapting to visual motor rotations a visual motor gain is a similar thing except when i move my arm six inches if there's a positive gain say of a factor of two the cursor on the screen would move 12 inches or if it was a negative gain factor if i move my arm six inches the cursor on the screen would only move three inches and it turns out humans are also very very good at figuring out what distance of movement to make in order to to make the cursor move where they want so we're very good at adapting to visual motor rotations and video motor gains so the way we apply these kinds of perturbations in bci is again through the population vector algorithm now remember in the population vector algorithm the way it works is that if you have a bunch of neurons he's aiming at a target over here he's recruiting neurons that have preferred directions and that way he's increasing their firing rates that leads to long vectors pointing that and when we average them we typically get a velocity vector that points towards that target if we decode correctly but what we can do is instead of having those neurons push the cursor in the direction that they naturally represent we could have those neurons push the cursor in a different way and so what we do is we take a random subset of those cells and we rotate the direction in which they push the cursor all right this results in two phenomena first you'll notice that the cursor isn't moving in the correct direction it's moving at a rotated angle relative to the intended direction of movement so that's the vision what a rotation we apply and the second thing you'll notice is now because the population that he's using is dispersed the average vector is shorter than it was before and since the average vector is proportional to the speed of the cursor what that does is it imposes a visual motor gain reduction as well all right so this is a very odd sort of perturbation to do why did we do this particular perturbation and the reason we did this kind of perturbation where we randomly chose some subset is that we wanted to ask the question if only certain neurons are contributing to error in this system can the subject identify which neurons are contributing to error and selectively just change their behavior or does he have to address the entire neural population as a group so we wanted to ask the question can the subject solve the credit assignment effect so we did multiple versions of this experiment where we chose populations randomly chose populations of either 50 percent or say just 25 percent of the neurons to perturb and we changed the amount of rotation we applied to their decoding directions so sometimes it was 30 degrees sometimes it was 60 degrees or 90 degrees we did all different forms of these and you can compute from first principles what the visual motor rotation and the visual motor gain reduction will be under each one of these manipulations and so we tested a bunch and this is what those uh those an experiment looks like in real time so what i'm showing you here is the control session we run beforehand to get the monkey familiar and just pushing the cursor around you'll see during control he's very very good at making straight movements towards that target his movement time is roughly less than a second which includes his reaction time now in a second you're going to see two things you're going to see that solid cursor which is what the monkey sees and you're also going to see this ghosted cursor which is what would have happened had we not perturbed the system the monkey doesn't get to see that but you'll notice that that ghosted cursor shoots straight towards the target meanwhile the real cursor moves off at an angle and the monkey is able to visually guide that cursor to hit the target so his success rates don't drop much during this experiment but he ends up making these trajectories now these are the results of this experiment after the subject has had 200 to 300 trials of experience with this and now what you'll notice is that that solid curse of the real cursor is moving straight towards the target and that ghosted cursor is the one that's missing so he adapts to these visual motor rotations quite well so this is just the average trajectories from all the experiments in which we took 50 of the neurons and rotated them by 60 degrees and you'll notice that those average trajectories point pretty straight to the target and we can assess the error in the movement the angular error by looking at the angular deviation from a straight line to the target at the halfway point at that dotted circle and so in the control session that average angular error is very close to zero degrees and the average time to target is a little bit less than a second after we apply that perturbation in the early part of the perturbation it's the first movement to every target we get these pinwheel effects where the cursor initially moves off in the wrong direction and then he visually steers it into the target the average angular error in that case was roughly 22 degrees and the average time to target increased to over a second and then after he uses that for roughly 1 200 to 300 trials you'll see that the angular air reduces and the time to target drops a little bit and then we turn the perturbation off and what happens when we turn the perturbation off is we get the opposite effect now he's used to working with this perturbed system and so his movements are now inappropriate for an unperturbed system and so you get this pinwheel effect that goes in the other direction and that also reduces over time and if we look at the learning curve in this system we just take the expected angular error which is that average error there and we plot that as a function of time we see that those angular errors reduce as a function of time where after about 300 trials now the error is down to about 30 percent of what it is on trial number one so he adapts to these visual motor rotations quite well what about the gain reductions well we can take the speed of the cursor and plot that as a function of what we expect the speed should be from first principles just due to the perturbation that we applied and what you'll notice is that most of these points are above the diagonal which indicates that the cursor is moving faster than it would be if he hadn't done anything to speed it up so this adaptation happens very very rapidly you see it already on the second trial of the system so he adapts very quickly to the speed reduction but that adaptation has a very limited dynamic range so remember i said we did several versions of this experiment where we applied different amounts of rotation which result in different amounts of visual motor gain a reduction and as we apply more and more visual motor gain reduction as the as the apply visual motor gain gets further towards zero these black bars represent what how fast he would have to push the cursor in order to return it back to its normal speed so how hard he'd have to push it it's the gain that he needs to respond with and the gray bars represent the gain that he actually responds with and he very rarely is able to speed this to push the cursor hard enough in order to compensate fully for the gain reduction we applied okay so that's what happens in the behavioral effects what happens in the neurons we can imagine three possible adaptation strategies to respond to this kind of perturbation so remember monkey's trying to push the target push the cursor to hit a target over here and initially what he does is he recruits cells that have preferred directions that point over here half of which we decode correctly and the result is a movement that's off up and to the right so one thing he could do to adapt to this is to re-aim let's say he aims at a virtual target that's located down here then he recruits the neurons that have preferred directions that point towards that virtual target half of which again we we decode incorrectly and that results in a straight movement towards the actual target so we call this compensation re-aiming it's a global strategy because it doesn't have to identify the cells that we perturbed it applies to all cells equally but it's not very efficient because when he applies this uh this strategy you'll notice that the decoded cells still are dispersed and so this strategy alone will compensate for that visual motor gain reduction there are other strategies you could use to compensate so for example if these cells here are causing noise in the trajectory what he could do is he could just simply stop modulating those cells with movement direction you could shut them down we call that re-weighting all right re-weighting and if he shuts them down those neurons will stop contributing to the population vector algorithm and that'll also result in a straightening of the trajectory all right so re-weighting is a local strategy he has to identify the cells that we perturbed and selectively shut them down and it's also not very efficient when the population of perturbed cells is large because if he shuts them all down he's left with fewer cells to control that cursor so it's only efficient when the number of perturbed cells is small but it's a possible strategy that he could use the third strategy he could use is something we call remapping all right and so the idea behind remapping is that if these neurons are forcing the cursor to move north what he could do is learn to just use those neurons when he wants the cursor to move north or set another way if he's trying to make a movement towards his target he needs to recruit the population of cells that have preferred directions that point towards that target and that we have not perturbed and he has to recruit the population of cells that have preferred directions that point south and that we have perturbed and he can combine those two groups and that results in the most efficient movement towards the target so this remapping is again a local strategy he has to identify the cells that we perturbed and then he has to learn how the algorithm is assigning them to cursor direction he has to start using them according to how that algorithm defines them so which one of these strategies does the subject actually is well he definitely uses reaming and the way we assess this is we take the firing rates of the cells and we can perform a maximum likelihood estimate to find the aiming direction that best corresponds to those firing rates and when we do that what we find is that if these black pluses here represent the target directions the blue ends of these arcs represent the aiming directions we measure during the control session before we apply the perturbation and those aiming directions are always very close to the target that's good that's that's how what we expect to be and after we apply a perturbation in the clockwise direction what we measure are that the aiming directions rotate that's represented by the red ends of those arrows they r arcs they rotate in the counterclockwise direction to compensate and similarly if we apply the perturbation the counterclockwise direction we see those reaming points move in the clockwise direction so he uses that reaming strategy but it's not the only strategy he uses he also uses that re-weighting strategy to assess this what we do is we measure the dynamic range of those cells those modulation depths in the control session before we apply that perturbation and then we measure them again after he's adapted to the perturbed decoder and we compare the sizes of those dynamic ranges now under conditions in which we rotated 50 percent of the cells it turns out there's no difference in the change in those modulation depths between the rotated group of cells and the non-rotated group of cells all right that's what's shown here on the bottom plot but for experiments in which we only rotated 25 percent of the cells what we find is that the cells that we rotate tend to have smaller modulation depths at the end of adaptation than the cells that we didn't rotate so he does shut them down relative to the control population now this is really interesting because remember we said that re-weighting only makes sense when the number of perturbed cells is small because if you use that strategy you're left with a smaller group of cells with which to control the cursor and it looks like he only uses the rewriting strategy when the percentage of perturbed cells is small and the final thing we did was assess remapping and the way we assess remapping is by measuring the preferred directions of these neurons after he's adapted to the cursor and we compare the preferred directions of those perturbed neurons the ones that we fiddled with to the preferred directions of the control population and the question here is do the preferred directions of that perturbed population rotate more than the control population and the answer is that across almost every one of the experiments that we ran that turns out to be the case it's a relatively small effect between 5 and 10 degrees but it's statistically significant in almost all cases and so the monkey is able to solve this credit assignment problem to some effect he's able to identify the the subset of cells that we perturbed and he's able to selectively change their tuning curves now the question is how much of the overall cursor error is is responsible uh is compensated for by these different mechanisms how much of that error is compensated for by these global processes re-aiming and how much is compensated for by these re-tuning things these re-weighting and remapping and turns out we can compute that by playing around with little models where if you were only to re-aim how much error would there be in the cursor or if you would only change those tuning curves how much error would be in the cursor and the answer is that roughly 85 percent of the error reductions we see during adaptation are due to these global processes re-aiming and roughly 15 percent still statistically significant but relatively small of those errors are due to those retuning processes over the course of 200 trials so we've now done an experiment in which we've held the same set of neurons for five weeks and we applied a perturbation on day one and we held that perturbation on we held it on for the first 24 days and then we turned it off and we just wanted to see what would this reaming versus retuning effect do over time and so what i'm showing here are the changes in preferred directions of the two groups of neurons the unperturbed group is here in blue and the perturbed grouping is in red and if you look at the difference in those perturb direction preferred directions that's that credit assignment effect he's identifying that perturbed group and he's rotating their preferred directions more and what you'll notice is that that credit assignment effect grows steadily over time it starts to saturate around 18 days in okay so that local retuning appears to accumulate over time to actually account for a significant portion of the error but you'll notice that he never actually asymptotes to the correct spot the correct spot would be where that blue curve on the top goes all the way to zero and that red curve goes all the way to the amount those cells were really rotated and he never quite get there gets there so to summarize the learning processes part of this talk subjects adapt very well to the visual motor rotations that we apply through brain computer interfaces but they don't appear to adapt very well to the visual motor speed changes and that's interesting because in natural reaching we adapt to those visual motor speed changes very well so there's something going on with the bci process which makes that harder to do and we see two neural correlates of this adaptation the first is this global process i described this re-aiming process which seems to be responsible for short timescale error correction or reducing bias in the system and the second is this local retuning process initially small but accumulates over time and this could be what's responsible for say the increase in linearity and tuning curves that we observe over a course of a month while the subject learns to use a device all right so that was the first part of the talk and now i'm going to switch gears and i'm going to talk we've talked a lot about sort of how this closed-loop system might differ from our expectations because the monkey's engaged in learning and now i want to talk about how we might be able to design a new decoding algorithm to help improve cursor control where we have to be a little bit more flexible about our design strategy have to think outside the box a little bit so the state of the art decoder in neuroprosthetics today is a kalman filter and the way a kalman filter works is you have two equations you have your observation equation and you have your state evolution equation so the idea is that we're trying to estimate the monkey's intended velocity of the cursor all right and we assume that our measurements the firing rates y are linearly related to those velocities that's the observation equation and part of the kalman filter also has a description of how those velocities ought to unfold over time specifically they ought to be smooth so the velocity on any given time step ought to be pretty close to the velocity in the last time step and what we do in the kalman filter typically is we assume that that's a random walk process where the velocity and this time step is going to be centered on the velocity the last times constant last time step with some noise around it now most of the approaches to improving this algorithm in the literature today focus on the observation equation focus on coming up with a better model for how firing rates relate to velocity that accounts for the non-linearities in the system accounts for the fact that neurons are poisson noise processes as opposed to gaussian noise processes these sorts of things i'm not going to talk about that work today i'm instead going to talk about our efforts to improve the prior model so it might be the case that these random walk models are not appropriate for how we actually certainly not appropriate for how we actually move our arms and they may not be appropriate for how we actually move cursors around on computer screens so here's an example this is a video that you might see in sort of any conference talking about neural prosthetics this is common filter decoding the movements look pretty good they're relatively straight to the targets the movement time is less than 900 milliseconds and which includes the reaction time what this video doesn't show you is that the subjects are actually really terrible at stopping they can move the cursor to the target very well but when it comes to stopping and holding the cursor on that target they do miserably so what we've done in this experiment we just increased the amount of hold time he needs for success and the amount of hold time on any given trial is given by that white bar and the amount that he he actually does is given by the slider that moves up and you'll see that in the majority of these trials when the monkey is using that cursor he's just blowing through the target all right and that's a disaster for when we're trying to really design a device that patients can use in the real world because what this means is that if i have a robotic arm and i'm trying to go up to you and shake your hand 16 10 out of 16 times i'm going to just punch you right in the stomach all right and these patients have enough problems without being known as stomach punctures right so the first question we want to ask is why is speed so poorly controlled in our brain computer interface devices all right we can think of sort of two explanations the first is that speed just might be poorly represented in the neural populations that we can record from that's a possibility the second is that speed could be in those populations it just might be encoded in some complicated way so that we're not extracting it with our algorithms and so to address this question we wanted to look at the speed representation during natural reaching movements the way we do this we have a rhesus macaque monkey he's implanted in this case with two utah arrays that we're recording from in motor and pre-motor cortex and he's engaged in this 3d center outreaching test so it's the task that i've been talking about he moves from the center of a workspace to a target on the outside except in this case he's doing it in three dimensions and what we wanted to do was compare how much information these neurons are carrying about speed and about direction of course this is a difficult thing to do because speed is a scalar quantity and direction's a vector right so how do we do it so the way we do it is we discretize both speed and direction into 26 bins 26 equally populated bins so what we do is we discretize it into entropy match distributions and then we take the spike counts recorded from our neurons in 30 millisecond bins which is the bin width we use during decoding and we just compare the mutual information between the spike counts and the speed discretized into these bins and we compare the mutual information between the spike counts and the direction discretized in these bins and when we do that what we find is that in individual neurons we get far more information about direction than we do about speed all right how do you like convey to the monkey that he has to hold at the at the you know is it possible that that that isn't become clear to the monkey yeah so what we do is we don't give him a reward until he's held for the right amount of time and so he's just got to hold there until he gets his reward so the fact that it's slightly more complex than the other jobs could that be one of the things that's biased in jerusalem uh possibly um but we've run it for several days and he never seems to get better at it and uh and so and we know he's trying to get better at it because of the strategies that he'll take so if there happens to be some kind of speed bias in the system he'll approach the target from the low speed side so that he can go through it more slowly so i think it's not that he doesn't understand what's required of him he understands it's just a serious lack of ability in the decoder yeah so when we do this mutual information analysis on individual neurons what we find is that they carry more information about direction than they do about speed individual neurons the question is how about at the population level and so to assess it at the population level we designed poisson naive bayes classifiers and we applied them to the population response and we saw how well we're able to predict uh which speed of the 26 possible discretized values in which direction that the subject that the arm was actually moving at and even in these population responses what we find is that we're able to correctly classify the direction about three times more accurately than we're able to classify the speed all right so what it looks like is at least in our normal population speed just might be not very well represented so this is a problem for decoding because even if we designed a coder to maximally extract speed the information just might not be there so what do we do now and so our idea was that maybe we can use direction to control speed we know he controls direction very well maybe we can multiplex the directional signal to not just control the direction of the cursor but to also control its speed the way we do that is by futzing around with the state evolution equation of the kalman filter all right so typically in a common filter right we have this it sets up the prior for the system the velocity that you're going to estimate on the time on the next time step is constrained to be somewhat in the vicinity of the velocity that's estimated on this time step and typically that matrix a we use the identity matrix which means the velocity ought to be centered on the velocity at the last time step well what we do is we apply a speed dampening factor and we make a a variable over time where it's proportional to the identity matrix with some constant that varies between zero and one now when that constant is one we get the standard kalman filter the velocity estimate is centered on the velocity at the last time step but when lambda is less than one that velocity estimate is shrunk the average value in your prior is going to be a little bit shorter than the value it was on the less time step and that what that results is in a slower movement vector on average and the way we set lambda is we take a page turns out a natural reaching movements when you go to make a curved movement you move more slowly than when you're making a straight movement it's called the two-thirds power law of natural reaching and so what we say is we compute the curvature in the movement by looking at the delta change in direction over time and the larger the curvature the decoded curvature the slower we make the cursor so that we make these highly curved movements corrective movements the cursor slows down and we make lambda sort of inversely proportional to that all right and then we tested this out online because and so the way we test it out we have that rhesus macaque implanted now just one utah array in primary motor cortex we're taking spike counts now from roughly about 85 neurons again in these 30 millisecond bins and he's supposed to do this center out brain computer interface interface cursor task where now the hold time is randomly distributed it could be zero milliseconds or it could be all the way up to 600 milliseconds all right and again he just has to hold the cursor on that target until he gets the reward when we do this with a kalvin filter this is what his success rate plots as the whole ad if the whole times are very short he's very good at this task but as those whole times get long he does very poorly this as well under 50 success rate but with the speed damaging kalman filter he does much much better so for hold times that are roughly around 300 milliseconds he's doing about twice as well as the kalman filter and what that oh so one trivial way to achieve this result is just if we slow the cursor down on average all right because if it's slower on average he ought to be more accurate all right and what we did is we quantified the actual movement time how long it takes him to acquire the target uh and which includes the reaction time we measured it compared it between the speed damaging kalman filter and the standard kalman filter we saw there was no difference so we haven't slowed the cursor down instead we've just pushed the accuracy up so this is what the speed dampening kalman filter looks like online so you'll see that the monkey has more ability to hold that on the target and a lot of times what you'll see is what he's really trying to do under both the kalman filter and this filter is that when he gets that cursor onto the target he tries to scribble it around to keep it inside the overlap area consistent with success and right now we're taking those natural corrective movements that he makes and we're actually using that as a signal that that cursor ought to move slowly and so in this case out of the 16 trials that we perform we get 13 successes now i should emphasize that if we use this new algorithm to try and predict natural arm movements so we record from neurons in motor cortex while monkeys are making natural reaching movements and we try and predict where the arm was at any given time it performs miserably all right the kalman filter significantly outperforms this thing offline and again what this tells us is that those algorithms that enable superior online control are not necessarily those algorithms that give the best decoding results offline all right so to summarize that speed dampening kalman filter portion of this work we found that direction is more reliably encoded in the neurons that we're recording from than speed is and that our sdkf decoding algorithm outperforms the common filter in closed-loop bci mainly because his corrective movements now act as a break on the cursor and we think that this captures a little bit more accurately than natural speed curvature trade-off that's found in movement all right more generally the result is that you can use these high fidelity neural signals like what's carried about direction and you can multiplex off of them to actually control multiple dimensions of a device both direction and speed in this case so some conclusions first off subject learning is the key to good neural prosthetic control all right all of our algorithms right now assume linearity and it turns out that's okay because neurons can develop linear tuning over time also subjects can compensate for any inherent biases and decoders and they do it very very rapidly within the course of a trial algorithms that predict our movements are not always the best algorithms for online control and to design better prosthetic devices we need first off a better understanding of the limitations in that learning process so this retuning analysis is very very slow are there ways that we can speed it up and he's very bad at adapting to speed gains in the system are there why is that can we understand what the limitations are there and work around and second off we need a more flexible design strategy we need to start thinking outside the box thinking about classes and biased algorithms that might actually work better than their unbiased counterparts that sort of thing so i've been throwing around the word we a lot in this presentation so i just want to there's the acknowledgements this is who i mean when i say we on the university of pittsburgh's side it's with andy schwartz group where i did my postdoc and people under hammer were responsible for most of the learning processes work on the carnegie mellon side i've worked most with rob cass and and for the speed dampening common filter work the bulk of that was done by our student matt golub and uh and his co-adviser byron neal and then our funding sources of course thank you is much known about how the brain normally does speed control so is it that there's another brain region which is representing speed better or is there some kind of piggybacking yeah i'm going to say not much is known about how the speed how the brain does normal speed control so probably the most thorough study of speed representation at least in primary motor cortex was done by mark churchill and krishna chinoy and what they found is that when you're making movements these center out movements at one speed you get one set of neurons that are active when you make them at a different speed you get an entirely different set of neurons that are active uh which is a a terrible scenario for our decoding um results said you need to learn more about the limitations of learning um i was curious in terms of identifying an upper bound for what you can learn like if you just totally naively plug some uh you know interface onto the motor cortex and help drop the brain would i may eventually just learn to use it yeah so it's a great question it's one we'd love to try the problem is that if he can't use it he doesn't try this is true for humans too there has to be some baseline level of success rate in order to even engage a subject in a task so you have to start them out in a ballpark and have them engage with it work with it develop some control over it we don't know what that ballpark means we don't you know we don't know why it has to be there we don't know if it's just because the subject won't engage with it or is it just because that first decoder is so difficult it actually can't be learned are there only certain classes of decoders that are learnable and certain classes that can never be learned we don't know and we're working on ways to try and address that question with offline estimation have you guys seen what type of accuracies you get so if you're using this linear model but if you're using non-parametric black box machine learning method what accuracy can you get yeah so people have tried algorithms of various complexity so the ones i'm directly familiar with are where we've taken that encoding model and and we've made it non-linear to represent what neurons actually do and we've put poisson noise models on it which are more accurately represent what they do and they work a little better five percent better maybe as much as 10 percent better but it's not the kinds of gains that you'd expect to see in the system now in terms of so people have done more complicated machine learning types of algorithms where say support vector machines just try and classify what the next actual intended movement is based on what the movement is on the last time step and do that rapidly enough to move trajectories through and do them in self-adaptive ways where the support vectors can adapt over time depending on which were more recently used and that sort of thing and again they work okay but they haven't led to vast improvements in control and it could just be that we haven't hit on the right algorithm yet i'd argue we need a better understanding of the system so that we can guide our search to the right algorithm yeah so uh if we look at uh surface level for example from recordings like easy and not uh by on neurons level is yeah so we can absolutely decode stuff from the surface level and people have done cursor control based on eeg it looks like for eeg signals it takes much longer to learn so subjects have to practice with the device for weeks to gain any kind of proficiency with it but it can be done and so we don't know why the learning for eeg is slower than for when working with single neurons it could be that the signal-to-noise ratio is lower or it could be that it's more difficult to synchronize enough neurons to actually generate a potential that's redoubtable at the scalp but yes non-invasive methods have been used and and will continue to be used i think and doesn't depend upon the complexity of tasks for example i would guess that i mean aiming a cursor extending your is just to do a handshake so if the complexity is less does it like affect i mean decoding yeah so one one thing that people have used eeg signals for very successfully is to control spelling devices and so it turns out you can make a category selection using what they call a p300 speller which is an eeg device and what they do is they flash letters on the screen until and and when you see the one that you want to use there's a recognition signal that's you can read out in the eeg and so then it selects that one and they can use spelling devices like that so these sorts of categorization tasks tend to be simpler than the continuous control tasks i've been talking about and there you can use eeg things but yeah as the complexity of the task scales up it looks like we need high fidelity control signals and so people have never used eeg control to get good control much above two dimensions and these days with neurons human subjects are actually controlling robotic arms with about 10 degrees of freedom so it looks like single neurons just enable more complex control yeah um so i have a couple of questions that may that may be naive but um it's the way i understand the the detection algorithm is you designed it to figure out what direction monkey wants to move um and in in order to get the mother to be able to stop at the target you've created a way to allow the monkey to sort of circle around or give or give it a give it a sort of artificial stock cue um and so the first part of my question is um if you take one of your decoders can you just get the monkey to maintain the cursor right at the origin where it starts or is it always going to be moving around and then the follow-up question for that is is there maybe some stopped signal or decrease in general global neuronal activity that your decoder might because it's averaging all of these inputs think it's just going to take the best best guess of the direction but really what the monkey's done is it's trying to sort of turn down all the neurons to a baseline level and it's just trying to tell the damn thing to stop now so is there is there a way that you could identify a stop signal and maybe use that instead or as a compliment to this sort of more elaborate speed control sure so to address your first question is it can he hold the cursor steady in the center of the screen the answer is it doesn't see himself so we've tried that we've tried not only peripheral targets but where he has to make movements from peripheral targets into the center and he's just as bad stabilizing in the center as he is at the periphery for your second question of is there some some signal about speed carried at the population level the answer is there might be but whatever it is it's not carried in such a way that a poisson naive based classifier can pull it out right and the only assumption really that the poisson naive bayes classifier is making first off it's assuming the neurons are poisson which is typically a pretty good assumption but it's also assuming that the neurons are independent but if that assumption holds true it doesn't matter it may not be linearly encoded or that sort of thing even if it's non-linear speed firing rate relationship that classifier ought to be able to pull it out and even then we're not able to classify speed as well as direction so it could be carried in this in the neuronal population but if it is it's violating uh any other sort of simple rules that that we can work with so there's another there's another interesting twist to this problem which i didn't talk about but it's the question is what kind of signal ought we to expect in the brain related to speed all right so should a speed command signal look like the speed of the arm so if the speed of the arm is bell-shaped should the speed command signal be bell-shaped and the answer is probably not because the arm is a non-linear actuator all right at the very least it's a low-pass filter all right so if you feed in whatever you feed in you're going to get at least a low pass filter out and the arm is actually much more non-linear than that so it could be that the command signals we're really looking for ought to have some different temporal shape than the actual speed of the arm and we don't really know how to extract that so speed might be in these neural populations but if it is it's in some complex form that we we can't access yet all right