Memory Consolidation During Sleep: Hippocampal Replay Mechanisms | MIT

[00:00:01]the following content is provided under a Creative Commons license your support will help MIT OpenCourseWare continue to offer high quality educational resources for free to make a donation or view additional materials from hundreds of MIT courses visit MIT opencourseware at ocw.mit.edu now why are we interested in sleep so we kind of think about this two modes online mode and offline mode the online mode you're taking information in the offline mode you're going back and evaluating information what's the purpose of evaluating information that you've taken in well if we think about the general problem that's being solved the problem of intelligence problem of intelligence is trying to is trying to understand and infer these sort of deep generalizable relationships rules you're trying to expect rules from instances and you'd like those rules to be Jenna's generally applicable as possible and in order to do that presumably one has to go back and evaluate the many individual instances to try to extract some kind of statistical regularity and then perhaps evaluate models that you have constructed in terms of their consistency with individual instances that you've already collected or perhaps future instances that that haven't yet play so you've got you get the you have the some of the raw material you bit a little model and then you continually test that model against new information that comes in and the question is when can you do that now you could do that when you're out and about in the world and well as as you as you saw and read in that paper when animals are sitting quietly they very quickly can switch into this kind of offline mode that looks a lot like sleep in fact electro physiologically sleep and quiet wakefulness and the hippocampus are nearly indistinguishable it's the same kind of offline mode it says ok when you're not and you know I'm not using this if the camp is not being used taking new information is in I quickly switch into this offline evaluation road but during sleep I no longer have the constraint of having to direct behavior behavior shut off inputs are shut off and now I can switch into this purely interest but internal introspective mode so what goes on during sleep in the simple experiment and we'll just we look at the activity when the animal is performing behavioral tasks and then we examine activity during sleep both before and after and ask is there anything about behavioral experience that changes activity during sleep and what we found was that if you look at activity during behavior and what you think about these spatial sequences being expressed and you look during sleep afterward you find that the spatial sequences are expressed again so the hippocampus replays the firing of these cell sequences but if but it replays the firing of these cell sequences at a timescale that appears to be compressed relative to the behavioral timescale so these are 8 play cells animals were walking from left to right the ticks indicate that's the location of the peak so these play cells will fire 1 1 through 8 as the animal moves along the track over about 5 seconds that's how long it takes animal walk from left to right the same sequence of 1 2 3 4 5 6 7 8 gets replayed during sleep but now over about 150 milliseconds same sequence so it's you're preserving time order not absolute time and that when you ask when these sequences are expressed what's going on in the local field potential these oscillations and this is where you find these sharp wave ripple events this is this ripple this is this ripple like event that I was describing to you so these sharp wave ripples are when these apparently compressed sequences are being expressed I can point out that a simple model for these sharp wave ripple reactivated sequences is the same model that's used that I like to use to explain phase procession during the fête oscillation that is I give it an input I sweep inhibition from high to low such that cells that again the strongest input fire earlier and this will reactivate a sequence so the difference between a theta sequence and a reactivated sequence is really just where's the input coming from if the inputs coming from what I'm actually experiencing right now now it's a theta sequence and as I move the input changes systematically and so again it looks like it's encoding information about space if I'm now offline and this information is being delivered to the hippocampus from some other source you can apply exactly the same operation get the same kind of sequence it's just now it's on information that is not tied to be sort of your you know immediate context or location so that's the only difference where is the input coming from now further if you think about this model you can imagine well thats of the depth of disinhibition could actually affect the length of the sequence but that's just sort of a mechanistic thing and if you actually look at the mechanisms that regulate sharp wave ripples in the theta oscillation it s it basically comes from the same structure it's a structure that regulates acetylcholine and that's a it's a neuromodulator that's associated with attention in memory and the structure called the medial septum so the medial septum provides the drive the cholinergic drive of the hippocampus damage the medial septum loss of cholinergic tone was one of the dominant models of nerd nerd degenerative cognitive and memory loss in Alzheimer's disease so what you find is one of the earliest indications of neurodegenerative damage in Alzheimer's is the loss of cholinergic tone and the systems that begin to break down are the systems that actually fall along this limbic pathway starting with hippocampus central cortex so as though the cholinergic system that regulates the expression of this oscillation in the hippocampus when it breaks down it leads to general memory loss in you know disruption and turns out the medial septum is also involved in regulating the expression of these sharp wave ripples so same system different modes one you know quiet one sort of active online when inactive offline same kind of modulation of excitability through an addition but that's the idea simple model sweep inhibition that gives you the sequences and then control the input you control the input to control the content you control the inhibition to control the structure of the timing those are the two things so the question is how could you control the input when we kind of thinking about what is the input into the hippocampus under these two conditions and as I mentioned hippocampus you've got the internal cortex Central cortex gets information across the brain is sort of the sensory Association our visual cortex auditory for this there all this information about the world converging on the hippocampus and then getting modulated and so you can think about let's look in for instance visual cortex so if we simultaneous recording video cortex and hippocampus we could see how these two structures communicate and when we do this in a simple task is like little figure eight tasks and I won't go into a lot of the details but one interesting thing that came out from doing this experiment recording visual cortex and hippocampus is that recording in the visual cortex when an animal is moving in space what you find is cells and the visual cortex have spatial like receptive fields similar to the hippocampus they're not a spatially tuned but here for instance r88 visual cortical cells and you can see that they fire they fire in a sequence they'll have different spatial receptive fields this shows where the where this one visual cortical cell likes to fire different visual cortical cells will fire at different locations so you when animals are moving in space you actually see sequential activation of these visual cortical responses and so if you have visual sequences visual perceptual sequences in hippocampal spatial sequences one question is how do those sequences how do those sequences relate offline when they annal for instance during sleep so you have sequences during sleep and the hippocampus sequence is varying sleep in the visual cortex and it turns out that those two things are actually they're correlated so in hippocampus plays out a spatial sequence the visual cortex plays out a visual sequence that corresponds to the visual responses at those locations so hippocampus plays out sequence of where it was visual cortex expresses the visual visual stimuli that were present along that sequence one thing about these sequences when we're looking when we were looking at hippocampal neocortical interactions is that the sequences are now at a much longer time scale and in fact this time scale which here is about on the order of about half a second to a second corresponds to another oscillation that is characteristic of sleep known as the slow oscillation so when you go to sleep brain rhythm the oscillations the dominant frequency start to slow down and you'll get this oscillation in this one Hertz or so frequency range that looks like if you record activity from a bunch of cells it looks like this this is the activity of a whole bunch of cells in the visual cortex in the hippocampus and you see that the activity is flipping between lots of cells active no cells active these are the so-called up-and-down states of cells during sleep that every half a second to a second or so many cells will become active then they'll all be shut off they'll become active again so you're flipping between is up and down like States and if you look at these up and down states in the visual cortex and you look at similar activity in the hippocampus you see these up and down like states in the hippocampus as well but you'll notice that the up state in the neocortex leads up state in the hippocampus so a neocortex first then hippocampus in your cortex tippecanoe clue neocortex seems to lead so again this question of who is providing the input you know that simple model or I'm sweeping inhibition during during the theta during active behavior that information is coming in from perceptions what I'm actually during sleep crisis where is that information coming from well this would say this is where it's coming from it's coming from these cortical areas let's say sensory visual cortical areas turn on they provide the information that the campus now the hippocampus turns on the campus responding to input coming in from sensory cortex if that's the case could we manipulate hippocampal activity by manipulating sensory cortex so this experiment simple experiment answer was yes need in this case we used the auditory system one of the reason for use the auditory system is unlike the visual system auditory system remains in a state of persistent and vigilance even during sleep auditory cortical responses even animal goes to sleep measure to measure auditory cortical response the same as when it's awake so essentially the auditory cortex stays vigilant during sleep which has clear evolutionary value so sleep you know it's trying to minimise arousal by you know sort of shutting off a visual input which may be of limited value anyway given you know of sort of the circadian nocturnal nature of you know of sort of visual stimuli so you can't see anything might as well actually close your eyes but you can still hear things so you and other animals are still listening if you hear you know pickup threats that might require that they actually work up so anyway taking advantage of that auditory system on so training animal on task where it learns to associate auditory cues with locations one sound right sound means go over here to get food left sound means go over there to get food very simple task then animal goes to sleep and you just continue to play the sounds so it's learned something and now you try to bias cortical activity during sleep and ask what does that do so this is the idea when the animal is actually running we can decode hippocampal activity so we can tell oh here's a hippocampal pattern that corresponds to the left side or the right side and in this case when the animal is performing the task play the right hand sound animal goes to the right hand side see this in the hippocampal response play left hand sound and I'll go to the left hand side again left hand plays fields and they don't was on the left right hand plays fields on it so this is this is what the experiment and the behavior looks like from the standpoint of the hippocampus very clear right right right sound right hippocampus right play cells left sound left play cells but now the animals going to go to sleep and we're gonna do the same thing continue to play the right sounds and the left sounds it's just that now the animal is not actually moving on the track and so we ask does that can we buy us the reactivated sequences that you get so there's the same thing now animals awake but now it's going to go to sleep and we just keep playing the sounds a little ticks there indicate the delivery of sounds every 10 seconds or so so we play a sound and now we look at the response the difference here is that animals not it's not running it's not behaving sound response now will decode the activity so here for instance we take a little short window here about half a second and what you see is this is this up like state multi-unit activity lots of cells firing decode activity and ask when you play the left sound what does activity decode to and here you see okay it's gonna decode into the left hand side so that's the basic basic hypothesis play left sound you get the replay of the left side of the of the track you play the right side you get reactivation of the right side of the track and that's that's what you get so when you play the left hand sound left sound bias activates cell play cells on the left side right side bias biases place also on the right and you can do the same kind of psychophysical experiments which has been done in humans either with with different mode different sensory modalities it's been done on olfaction it's also been done in audition so the equivalent experiment using auditory cueing in humans where you have people learn the simple task this was done in Ken Taylor's lab or they do the simple spatial matching game it's like you know where's the cat card where's the teapot card the variant here was when they flipped over the cat card they would play the cat sound meow they flip over the teapot sound they play a little you know associated an auditory cue teapot whistle and then people go to sleep and during sleep they would play either like the cat sound or the teapot sound and they found out when they would wake up and now they do the task they're better at if they played the cat sound during sleep they were better at remembering locations of cats then than the whistle so this says not only to sleep actually contribute to memory but it's selective and it not only is selective but it can be influenced it can be biased you can direct the nature of memory processing and then our experiments suggest that well one of the one of the consequences of this kind of sleep manipulation would be bias the memory reactivation in the hippocampus and so the idea is simple and that is that cortex bias is the state that the hippocampus gets and then the hippocampus takes that state sweeps inhibition replays a sequence and that sequence thing gets played back to the cortex so you know the cortex it sort of knows you know if it has these sort of discrete states but doesn't necessarily know what the cause of correlations might be what happens next it doesn't necessarily know what happens next it knows what happened hippocampus knows what happened next it has lots of instances of that though well I saw you know you know I saw red light what happens next oh yeah you know I saw red light and all the cars stopped okay that's great you know if I'm just the cortex that's what I learned all the cars stop if I'm hippocampus I'm a little bit smarter than that you know you know last Tuesday I was there there's a red light and all the cars except for these cyclists man they didn't stop they just kept on going so wait a minute so this there's a rule you know red light cars stop bicycles don't stop so you were fine right you have to refine these various what appear to be simple rules prefrontal cortex often that's actually used in an example how do you use the prefrontal cortex Oh red light means stop green light means go that's great except in the real world that rule is too simple it has to be refined has to be refined based on your particular experience in fact if you're really sophisticated red light means cars except for cabbies will stop right if I see a cabbie guarantee that guy's not gonna stop using an accelerator right it's a and that's kind of information that's what the hippocampus has and so you imagine that's what's going on neocortex in each one of these slow oscillations saying red light and they pick Amba says oh yeah okay cars stop red light again oh there was a bicycle thing bicycles continue to go red light and that was the cabbie incident so now you have all these different these sort of causal sequences that are being expressed back to the neocortex presumably in order to establish this this more comprehensive consolidated model of real-world traffic lights rather than the cartoon traffic lights and so that would be the idea what's going on during asleep now during quiet wakefulness you can think of the same sort of idea and that is that you're kind of evaluating evaluating multiple calls of contingencies each of which might be expressible as a simple rule but might also be experienced as distinct variations of that rule and the ideas okay do we use the rule or do we use the exceptions or how do we actually you know how do we refine the rule based upon these different instances or exceptions so you can think about that is refining the rule that's like the learning sign applying the rule that's the sort of a memory of decision-making aside so you can think about during quiet wakefulness this reactivation being used in the service of actual learning or in decision-making and so again you're at the paper and one of the things that we had discovered in interesting about reactivation during quiet wakefulness was that when animals stop after running on a track they do reactivate sequences this is raw data animal running from left to right and then stopping for a long period of time and you can see activity there these bursts of activity you blow things things up these these sharp wave ripples last about half a second or so so you see a burst of activity you see these place cell sequences in this case the sequence actually runs in time reversed fashion so in this case you know time reverse fashion just doesn't this seemed like planning or decision-making it may be a valuation of temporal correlations that might be relevant to behavior and learning but what would a reverse sequence actually have to do with with you know spatial learning and memory well that actually came from the insight into how this might be used came from computational models in fact computational models that the postdoc in this case Dave foster who made this discovery had used in his doctoral work where he was actually building model reinforcement learning models with spatial navigation and one of the problems in reinforcement learning is that reinforcement often comes after you've actually carried out the steps that lead up to it in other words you walk from left to right you get rewarded when you get here what you want to know is not just this is where the reward is what you really want to know is what were the what are the things that actually led up to that notice I want to take credit reward value and I want to spread it backward in time to place value on the things that predict or led up to reward the so called temporal credit assignment problem how do I give credit to things that actually lead up to a predict reward and thinking about how you might solve the temporal credit assignment problem this reverse reactivation actually has the capacity to to solve that problem in one step if you imagine when animal gets to the end gets some reward and if now at that point I pair the delivery of a rewards signal which I will indicate here is this is a cartoon suggesting this is dopamine or reward signal and I'm gonna pair that with the reverse sequence and now you can think of the Association this is my current location this is way back where I started from current location gets paired with strong reward remote location gets paired with low reward so this association will essentially solve will convolve this sort of this reward discrete reward impulse function will turn it into this continuous graded sort of monotonic reward gradient function translating this into this essentially in one step so the thinking is hey animals are actually using this to learn and they're using it to solve the temporal credit assignment problem in a way that would be that would be important for general reinforcement learning now I won't go into so we actually did this experiment recorded from the reward area the VTA and the hippocampus and indeed during these reactivation events you see the pairing of this reward signalling and it's not only do you see the pairing of the reward signaling but you find pairing of the reward signaling in which the precise firing of reward of source signals map on to the delivery of rewards at goal locations so it's not just certain sequences are good others are bad it's that there are certain locations along the sequence that have differential reward value so there's a mapping of relative reward to location and that if you look at this while animals are actually performing a task you can see biases in these sequences that correspond to planning so both of these elements of sequence reactivation this was some work by Dave Foster or he looked at an animal that has to just forge and find a location space when animals are searching for a location and you look at these reactivated sequences it turns out that reactivated sequences tend to be directed toward the locations where the animal thinks the goal might be so it's sort of thinking about things that would lead to reward so both sides of this kind of sequential computation seem to be expressed in the hippocampus Co expressional with reward during evaluation or learning the expression during reward directed behavior in the service of planning a decision-making and then finally round was done this is the paper that we had that we had that you had read is just looking at the structure of these reward sequences on long tracks and the thing the sort of the most salient point that came out of this paper is that this shirt this the phenomena of sharp wave ripple sequential activation that corresponds to these theta sequences when animals are sitting quietly over the course of these longer upstate like events takes the form of bursts of sharp wave ripples or sequences of these short sequences so there's this compositional notion that you form long sequences out of little short sequences and there the question is how do you actually put these short sequences together and why would you actually have a why would you have a representation that has this kind of compositional structure or you can it's the Lego block kind of idea where I can put together sequences from these fundamental ease more sort of elemental sequential units and this is just I don't show the movie but if I actually zoom in on one of those sequences that in that particular instance with animal stop you can see that longer there's a longer reactivated sequence that actually takes the form of for shorter sequences that's how that's how long sequences are being evaluated and I can't show that and one interesting thing about this was that if you look at these different sequences at different lengths different lengths shorter sequence longer sequences they all seem to have a fixed velocity and that is this is kind of a fixed where it is the speed of thought so that there is some fixed constraint that that that evaluating you know further out and the future simply takes more time and so that suggests that there's an interesting there's an interesting constraining capacity for extended evaluation that comes in the form of these like these these oscillatory modes or the slow oscillation longer or slower frequencies give you essentially more time longer sequences and that potentially if you want to go even longer you might actually even link sequences across subsequent cycles just as you as you could do across successive sharp wave ripples you might be able to do across successive slow ways so the idea that you have these oscillations that you can link or couple sequences across different cycles of these oscillations that you have oscillation of different frequencies suggests that that's the moat that this is the mechanism it's a cop no mechanism for sequences that are nested within oscillations by combining these oscillations you can construct these sequences in ways that presumably contribute to cognition but we don't so we now kind of see how things are structured and now the question is can we actually manipulate it so that would real that's really the challenge seeing the compositional structure having so demonstrated potential access the ability to bias content to core to bias the you know the structure when things occur what actually occurs now I think the capacity exists to essentially engineer the sequences themselves and that is that during the sleeper quiet wakeful States get animals to think about things that they have not actually experienced and test the hypothesis this is what actually this is what animals are using to construct these models to essentially tinker with the blocks of memory and cognition and then finally just pointing out its interested these ripple sequences same space and time scales theta sequences so again it's suggesting this is the fundamental unit it's not unique to the hippocampus essentially any brain system could express this kind of structure using that simple model where you see ramps and you see oscillations you're gonna get sequences and this capacity is something that probably is broadly expressed and enjoyed by different brain areas so there you go you