Representation in Cognitive Neuroscience: Lynn Nadel & Andrea Hiott

[00:00:00]and uh you know I'm I'm going to give a talk uh next week probably on you know sort of the early history of work on the hippocampus kind of the background to the emergence of the cognitive map Theory kind of what the field was like in the 50s and 60s and 70s and you know how how it all came about you know how from my perspective you know all the pieces came together so to speak historically Edward that's gonna be interesting you gonna film that or something I'm don't know I don't know somebody will probably uh you know Edward said you know the students here they don't know anything that happened before more than like 10 years ago said it's a long time ago when you're 20 yeah so last yeah right so last year I gave you know three three research talks here of one sort or another but this year I I don't have three in one year you were there that much last year I not one month I gave three over the course no no I think I gave two it doesn't matter I gave yeah I gave my research talks this year I don't have anything new to say about research specifically that I wanted to talk about so this is this is going to be my my presentation while I'm here it's a a history talk which I'm working kind of fun putting the pieces together and trying to trying to reconstruct the causal the causal chain you know that gosh I'd like to at least read that if you don't publish it somewhere because that's that's probably very interesting to see how you put that together yeah well if someone much of it flowed through Montreal and you know and I know we talked about it yeah yeah right exactly yeah this is the history that I've been talk to you about anyway so speaking of history today we're gonna try to talk about this really difficult word representation dare I even say it but um yeah in in terms of cognitive Neuroscience though so for all the people who might be thinking we mean it in other ways because you can talk about representation in so many different ways whether you're in art or philosophy or whatever but we're going to talk about a little bit philosophically I think but mostly through sort of cognitive Neuroscience right so I guess I was I was thinking today I just kind of play like the student instead of trying to pretend like I really understand all this because I was looking at your um your class which I was just trying to find but yeah in any case there's a class you taught like a a long time ago right couple yeah it's like 10 years ago I gave that you know most of the stuff that's in there you know hasn't changed because a lot of no no not at all and even some of it I realized had become kind of popular again like when you were sort of well we'll get to it questioning certain aspects of what is representation and you need a perceiver to see the representation right in any case um before we get into all that let's start with just cognitive Neuroscience like you do because I was thinking you know like it's really interesting right and I think for people who are trying to follow this like why do we focus so much on the the brain and I think in those notes you kind of start with it's it's been that way forever right like what are we why are we trying to provide a brain-based account of of of cognition um I mean is it really just like simple like okay we just feel like we're in our heads because all our sensory organs are so located there you know yeah I mean if you were I mean if you took if you were to take a kind of a a Barkley stance and John actually did this at the dinner he was being somewhat upstrap and he said you know there's no world out there right I mean it's all constructed in our head basically and so I mean if you take that kind of a position I mean it's all inside our head and we're we're creating this whole external world you know which may or may not match up to what's actually out there I mean who knows uh it better match reasonably well otherwise Evolution would have you know squashed us but yeah so I mean I it it it's inevitable that you know you think about you know how how is it that we understand and comprehend the world so that we can behave so that you know our so that our Behavior which whatever one thinks about the notion of representation it's hard to it's hard to say that the brain has nothing to do with behavior right I mean it's got it it's what makes us move you go back to movement right so I mean we move because our brain you know tells us to move and and we move the ways our yeah but we also have a body and we I mean it's not disconnected from the body so I guess it's just like you move into the Mind Body problem but is it it's just this feeling we have right of our experience of the world feels like in our head doesn't it we can't get away from that it seems to me no matter how no matter what philosophical position we take so so with that you know as a as a starting point you you have you have to think about well how how does it come to be that the brain models let's use that term and you know represents you know the outside World in a useful way you know so that it can guide you know guide our Behavior going forward there's really doesn't seem like there's any alternative to that position so it's not you know people think about well so what what ises how does the brain represent the world how does it how does it do that you know there's no other way to there's no other way to think about you know strikes me as a scientist there's no other way to to to think about behavior um yeah so if we just think people coming to kind of awareness of themselves then you have this feeling like you're sort of in your head and the representation there in that really early sense is just that it's almost like we we can sort of have these images of ourselves or we can sort of hear our thoughts right and so that feels like it's some kind of we're representing the world in our heads right in a very simplistic way yeah you ask if you ask a five or six-year-old which I've done this with my grandkid and you know you to ask them you know where they their thoughts come from where they you know it's in they point in here it's in here oh that's fascinating yeah exactly I mean they they understand you know it feels that way exactly as you say so we so we I mean but there's okay I mean I can leave it at that except to say that there's you know there's a couple of different facets of this representation problem I mean one of them is how does it do it you know and how does it you know how how do we represent the world and how are those representations used all the kind of mechanistic questions about it but then there's the more complex question which I don't think or I certainly haven't made any progress on which is you know why does a particular kind of brain activity represent a particular thing I mean why is it that you know activity in visual in neurons in the visual system actually represent visual phenomena I mean it's that it's the kind of the hard it's the Consciousness hard problem but it isn't just about Consciousness it's simply you know how come certain forms of activity represent certain things kind of that's the Deep question you know for me right um yeah well I just I think it's it's like we all have this fascination with becoming aware that of our own thoughts in a way and then they don't feel physical right absolutely they don't feel physical so that's a very we're physical and we're experiencing them physically but from that vantage point it feels like something not as physical as for example just to touch the table right so like your little five-year-old uh grandchild you know it's kind of feeling about it yeah and um yeah yeah but so there's just Mind Body problem of how can this happen right how can this physical body or this brain give rise to that sense that even your 5-year-old grandchild has of self right it feels very mysterious it is very wonderful right we get confused mind is it Consciousness all these words we use in so many weird ways right but when it comes to trying to study the brain um you know like actually thinking about what is reality so to speak right that becomes like if if we just I think I don't know how you feel about this but I think it's a very bodily experience of of cognition and representing the world of course the brain is Paramount but let's just say we're going to focus on the brain and how this mental experience arises what the brain's role is in it and then we bring in this world represents right so what we're trying to understand is different kinds of reality isn't that how you say it in your talk like there's a kind of current reality right now where I'm having thoughts and then there's this thing we call memory which is the fact that I know there was a reality yesterday or that you were the states recently now you're in Norway and all of this and then there's the future right what you talk about right as like right prediction we can the brain can do all of that you know and and but but of course mostly we get to describe what we're thinking about in words right but we know for from a variety of perspectives that the words we end up using separate from words are form of behavior of course but a very specific form of behavior uh they never quite capture what it is we're thinking about I mean you know we get as close as we can but you know any anyone who's kind of thought introspectively or Tred to think deeply introspectively knows that they have some thoughts and they're never quite success they're never 100% successful in getting exactly what it was they were thinking into words words are really good they're better than Grunts and and but you know they still don't quite capture the nature of what it is of thought so thought goes beyond language I guess is the is seems clear so whatever however is it clear I think that's really hard I think it's really hard I think it's clear I think I mean obviously there are images and there's good evidence for this actually Nancy kanwisher and dar sa talked about this in their can in their C talks there's pretty clear evidence that you know that you can dis you can separate out language from thought I mean many thought a lot of thought is language but not all of it so so you know that means there's some deeper so the representations that we have in the brain are not just words there's no kind of lingua franka there's no fedori and what's the language of thought then it's not words so the language of thought has something to do with this issue of representations you know how is the I mean I I think it's clear if you've read all these papers and studied all this but for just people who haven't you know it seems like your thought is connected to either words or images is so I don't know if you have an example or like easy kind of way to show yeah but uh anyway I mean I think the data certainly from the data suggests that the two are not totally uh overlapping and that again just points to the you know there's some somehow the brain is representing information in a way that is useful that you can use to generate language that you can that you use to generate behavior and it must match the world you know close enough so that you know you survive in the world and the species survives so to speak um and and uh so how does it do that that's that's kind of the the core question you know what's you know and the so the earliest default thought was well let's look at Vision you know you say okay we know that the an image of what's out there in the world you know is kind of reflected upside down in the retina well okay that's it problem solved you know it's just like a little movie playing in our heads obviously we have a picture you know we have a picture on the retina and we have this right and it's this movie playing basically and and that's how we represent the world and of course that's complete nonsense because you know that who's watching the movie right you have this infinite regress of a little person in your head watch watching ex mle screens yeah but of course people thought that for a long time they actually thought you know that that was a plausible story basically and that's how it feels again you know back to the five-year-old experience that's how it feels yeah and and that's one of the reasons why for for decades maybe a century people were all twisted up about the uh about the fact that the image is inverted on the retina it was like wow this is a serious problem how does the you know how do we deal with the fact are we watching everything upside down all the time like you know and you know you can find hundreds of papers probably you know written trying to you know address this you know the really hard problem of the of the inverted retinal image and how the hell does the you know does the brain solve it it's completely meaningless of course because there's nobody inside looking at the at the image on the red that's got nothing to do but we just take that for granted in a weird way so long so that picture in the head kind of story that you know that I think there was called an idola or simulacrum I mean there were words that people put you know that you just the brain just creates a replica of what's out in the world inside your head but it but it completely in the end becomes pointless because this default you know to the homunculus you can't get away from it that problem doesn't go away by the way no matter what theory you have there's always a problem of a decoder I mean who's who or what is doing the decoding if any if anything I mean there are some people would argue that you know there isn't a decoder it's all just a continuous loop between the environment and the and the Brain basically you know and and there isn't a single decoder it's all just connected to it's this kind of that's the embodied cognition approach essentially where everything you do is immediate sort of coupling to the and there's immediate coupling and you don't have to think about a decoder because it's all part of a and you know by the way that may not that that may turn out to be a pretty good story in the end yeah yeah I I think it's part of trying to figure it out but it's also like that's kind of the question of representation in a weird way too is because like as you say I think in those slides um or you've said in other work you know like we often do that same make that same mistake that people were making for so long of thinking there's someone in the head but we forget that we're the scientist looking at the data that we've just gotten from the B uh brain so we forget that we're decoding it somehow that step gets like taken out right so it's yeah yeah yeah it must be in the brain in the same way it's in this little image of a of a screenshot or something that we've taken like oh it's but that gets confusing too doesn't it that the fmri or the or whatever we're looking at isn't actually in the head like that that's exactly right I mean it's so we as experiment is looking at you know unit recordings or fmri or you know multi-unit recordings or these calcium what whatever we're looking at you know and we and we connect up certain patterns of activity in the brain to certain behaviors right you know we just assume that those patterns we see those patterns and we see the fact that they're connected to these things but inside the head how is that happening basically I mean the the inside part of the head is not looking at the data the same way we are we're seeing all the data right more right any given part of the brain is seeing only a fraction of the data so to speak that we can look at as experimenters and we jump to conclusions as experimenters oh well you see this pattern of play cells firing it means you know such and such well you know is there any part of the brain that actually con seeed that pattern of play cell firing in the same way we can see it and use it in the way that we think it's being used or not uh not probably so not I mean even using language to talk about it is weird because of course they can't like observe it in the way that we do with our senses or something right right maybe there's some kind of pattern sensitivity or something I don't know that could be what's happening I mean what's happening now it seems I mean all the that lecture kind of brought you know ended at the point where people were saying well look what the brain obviously represents information in some sort of a distributed code a population code that's that is generally you know accepted there may be accept you know rather unique exceptions where a particular cluster of cells you know has a very unique function and you can say a small number of cells you know do the whole job but mostly it's vast arrays of neurons in some population um what do we do with that at this point I mean again it comes to the issue where's the decoder for that for that for that population and what how do we connect up the the population story which is what everyone talks about now to to this issue of this higher level issue of representation right but and that's what's happened in the last 10 years so all the stuff that I didn't you know that I now think I'm just not expert enough about but which I hear hear a lot about here and which I heard about in the cley talks especially Daris sou uh you know it's about this it's about the kind of structure of those of that population of the and you know people are talking about the geometry of the of the structure of that population and the possible phography yeah etc etc and that's kind of where the field is headed now and that's outside my zone of comfort that's why I said I don't know that I L more to add now because I'm not sufficiently uh I'm not I was I would say not sufficiently sophisticated but I could say I'm not at all sophisticated mathematically to understand you know the kind of stuff that is being done now that points out the kind of consistent geometric uh and these are not this is not clustering or topography in the way that one sees in the visual system this is simply the kind of possible states that this multi vectorial kind of collection of cells can take you know that that somehow serve as the way it represents say an object or something Kenneth Harris was here just the other day he gave a talk on rep what you what I would say was like representations in the visual cortex and it was all about you know the the population code and and the bottom line of what he was saying was that if you that the ex the excitatory neurons the population code in the excitatory neurons is very high dimensional and captures the specific features of the sensory input of the visual input the population code of the inhibitory cells is low dimensional and reflects kind of the context you know or the modulation of of and those two things kind of playing off each other basic you know you you might want to check out Kenneth Harris's work on this stuff it's really he's a he's at UCL um okay uh so you know the folks here in Trondheim are on all are kind of very much on the issue of this the torus that you know the the manifolds right so the as I understand it a manifold is a kind of is all of the possible states that a representation can take let's say in the grid cell population you know the the there's a there's there are constraints on the on where in the state space the population can be right it can't be everywhere it can only be on this Taurus because of the nature of the of the and the Taurus you know which kind of constrains the possible States the population code can take IE you know how it can represent whatever is representing that shows up what they're showing here now is that that shows up by day n nine or 10 in the ratp before they are moving around in the world before they've seen anything their eyes are still they haven't explored there's been no there's been no they've had no experience yet but the pre-wiring in the system at this Taurus kind of con of you know sort of reflects the manifold of the of the cells that are available and it's already there by day 10 so so what that tells me and I think what it tells the Mosers as well is that some aspects of the way we represent the world are are given so to speak prior to specific experience although specific experiences can subsequently influence the way this thing works the kind of is really pure K you might say the Conan the the built-in structure you know constrains the representation and you know that's where the grid that's where the grid cell kind of property comes from it comes from whatever that constraint is that you know kind of determines the possible states that the population can take you know as a collection so I don't understand any of that stuff I mean I hear it but I don't intuitions about what that tells us about how the brain represents I mean that's going to be a part of the answer to the story of how the brain represents so it it goes One Step Beyond this notion of a distributed code I mean so you know you to step back into that lecture I mean I went from talking about you know the retina to talking about these to the topographic maps that what finds in the visual cortex again it doesn't matter that you know the the input to the to V1 is arrayed topographically from a from a representational perspective from a cognitive representational perspective again nobody's looking at it right but the fact that it's done topographically has obvious uh computational advantages I mean it you know things that are near each other may be part of the same object for example you know pixels voxels in space I mean so to speak I mean there's there's some real advantages to having a topographic code represent especially the spatial aspect of things you don't get this in in in the auditory cortex you only get this in the visual cortex you get the somata sensory cortex and so on you don't get it with old faction you don't get it with taste you don't get with audition because they the the it wouldn't serve the same purpose there you're not talking about things that are nearby in space let's um let's let's like Define this a little bit because this is a really interesting part of the history of trying to understand cognitive Neuroscience in the way we just brought up and getting to what you were just describing of this population coding now and thinking more in terms of even maybe that embodied idea of something ongoing that we're representing and so the brain is not necessarily representing but before we get there the topographic map I think is a great place to kind of like sit in this for a minute because so we started with we have this feeling of mental physical internal outter and then external and I think you talk about Plato and Aristotle and this kind of debate is it internal is it external yeah um and then we get to dualism and decard and but really what we're trying to figure out is like where what's happening here is it was something was just all in her head from the beginning in in terms of the way you described the the rats that are born right like is everything like that given to by God this this is the this is the argument that Baki is basically has joined I mean this the Inside Out versus outside in approach I mean is still either or which I'm against the either or but absolutely it's got that is right that that's clearly right so I think I may have already told you that that Mary Peterson and myself and Drew Mau and Sarah Burke in University of Florida they were former students of of Bruce mcon and Carol Barnes we've been working for several years now just slowly working in working our way into trying to spell out a hybrid a hybrid between the inside out and outside in approach to thinking about how the brain represents meaning you know I me it's essentially that the question we're talking about today uh we have made very little progress so far oh I actually don't think we've talked about that yet and that really interesting that that's G to have to be another subject I need to like read up on that because I I really feel like holding the Paradox that's how I say it philosophically is like I think the hippocampus is like trying to help us do that right because you can't go in either Direction all the way you need both so how do we do it and it's it's hard how to think about you know the fact can't be one or the other it can't be an either or somehow right or it's both I mean yeah it's like it's it's a new way to have to process this I think that's it's that I think all this research that you're involved in at least from my perspective is showing us a way to do that but it's very hard because you have everyone in their own camps trying to kind of prove and that's part of the process itself but I don't want to get too distracted away from topographic maps yet because this word map is really important and I think it can kind of like lead us because and there's this very cool thing in case people don't know the um is it Penfield who came up with the sensory or the sensory homunculus was it kind of mapped it you've probably seen it like a picture of the brain and there's this weird creature yeah that's the that's I'll show a picture of it but basically what that's showing is that in the brain um most people's brains there are certain areas that are associated with certain parts of the body right you can sort of it's not like map is a tricky word we could get into that but more or less within the brain as it's developed with the body there's a certain part that's get got a lot of attention towards for example the hands or the lips or or so well it's I mean what we're know what we're finding out and that was what the Cy prize was about was the face systems it's it's usually more than one area I mean the sensory homunculus that Penfield pointed out you know would make you think that you know there's this one place in the brain where the face is represented well no actually there's about seven or eight face patches it turns out you know of of increasing degrees of complexity you might say it and and increasing degrees of invariance when you get to the sort of top of that hierarchy you you have a system that is representing a face no matter which angle you're seeing it in and so on I mean so so but basically yes the the brain has these somata and motor I mean the motor systems the same right and that's so important what you just said because I think even in a lot of ongoing neuroscientific study is still taught in the old way not in the way you just said and that's a huge big difference right that there can be many different places where this is happening but it doesn't change that's the either or too it doesn't change that actually there is associated regularity right I mean this is it's just not a map like in this way we thought that's the localization kind of argument basically you know are is the face system localized to one part of the brain the answer is no we now know it's distributed in you know into several but within but each of the patches may have quite a very specific function it may be specialization within each of the face patches it's not as if all these patches are equivalent you know and they're all doing the same and collect and it's not yeah it's not like they're changing all the time so I think like to try to connect it to what we were talking about before you know we come into we're trying to understand how the things that we are aware of or experience are related to life and how the brain is doing it and um so for example with this map you can kind of understand that certain sensory input with certain part of the body course like if you hurt that part of the brain for example in that individual then it's going to affect that part of their bodily experience of the world and movement undoubted exactly right and all the that's the connection yeah yeah that's the connection but things like so on are all you know relevant to this and ramachandran has done some really lovely work along about this kind of stuff right so what do we make of that though I mean again that's a you know that that's sensory humulus at Penfield by the way he's part of the history lecture that I'm putting together because P it even goes back to Migel too you know exactly right he he was a critical link between uh the neurosurgeon who did the surgery on HM and Brenda Milner who ended up studying hm and Sue corkin Penfield was the was the critical link he brought in Milner right or he brought well he he brought in Hebb who brought in Hebb who brought in milon right brought in Milner who brought in Cork who sent corkin to work with Milner on yeah Tang pfield is part of the story which is kind of cool yeah right so that I mean that you know that too is a little bit like the you know image on the retina I mean we don't you know yes the the brain is organized in that way that there are these topographic maps so to speak they they're distorted visual system there's even Distortion within the visual system there's you know asymmetries and all kinds of eccentricities and so on it's not a kind of a flat map you know that accurately represents the 3D structure of what we seem to perceive as ukian world that's not the way exactly but it's roughly topographical because that there's real computational advantages to that and the same must be true in the somata sensory Maps there must be there there's got to be computational advantages to having you know those things represented in neurons near each other I mean for start then you're then this becomes a wiring issue right and I think I mentioned it and I don't know that I mentioned it in the lectures but I probably in the chapter that I could never that I didn't manage to get the 198 chapter you know the the the computational advantages of of of having things close to each other involves this sort of minim minimum wiring constraint you know which Evolution presumably prefers you you you want to have things that need to talk to each other as close to each other as possible you also need to have long range Communications because you know Etc but to the extent to which you can have nearby things uh that you know that are functionally near each other that you know that's that saves extra wires which saves metabolism which you know yeah and by wire are you're you're talking kind of like heavan syntic what do you mean by wiring that's separate issue no this this is just an issue of of development I mean when you're when the brain is developing I mean it it takes it takes a so they're literally connected with by synopsis or how do you takes a lot of energy to to to make connections between neurons and the long and the the greater the distance between two neurons that are meant to talk to each other you know because they have some similar some similar function the more energy it takes more electricity yeah so you're literally talking about I'm really talking about connection the near in space yeah The Nearness in space you know is a is a big Advantage from a you know all other things being equal it's a big advantage to have things near each other if they're doing if they're doing something similar that's why the absence of any topography in the hippocampus initially struck people as this is so crazy basically have the neurons that represent two places in the world that are really close to each other in the world they may not be anywhere near each other in the hippocampus they could be you know it's random you know it seems like you know there's been some argument about that but so the hippocampus was this non-topographical map that's what made it so kind of cool initially in the context of what we knew about visual Maps here was something that you know it's not only visual we it was known early on that it was more than just visual but you know most of the experiments you know default to Vision you I mean the rats moving around the world that sees what's out there blah blah blah and yet the way this the way the brain mapped it wasn't topographical so and that I think that's part of that either or holding the Paradox coolness of the hippocampus 2 that we that everyone's having to deal with that but this is very interesting because let's like Okay so this this idea of top Topography is yeah quite amazing right like this was an amazing thing to figure out that within this organ in our body yeah probably different for everyone but as you develop probably you come with certain regularities that everyone shares but as you develop certain parts of your brain are going to correspond to certain parts of your movement and your sensation and the body parts right and that's fascinating it's so cool and then we thought of that as a map right that you can map this onto the brain and this is where the representation gets confusing because topography doesn't solve the problem of representation it doesn't solve any of that stuff we were talking about but it starts to seem like now we have a representation but actually we've drawn a diagram which is the representation of kind of being able to create another third thing which is a kind of map of like okay this part of the brain corresponds to this part of the body but then like we were saying people start thinking there's a map in the brain and this gets kind of confusing exactly right exactly I mean that it solves no problem at all I mean it solves the mechanistic computational problem clearly but not the meaning problem that's how put it in but not the meaning and representational problem so I mean so a lot of the work that is being done is really focused on the mechanistic problem I mean how is the system doing you know creating these maps and how are these things being and how are these things used and so on but and how do we know it like how has this experience that right like yes we found that wonderful correspondence and that tells us something but to then think oh well that's what's happening that's how we have this little movie in our brain no it's too it's too confusing but you know not we don't have this little we don't have that and I don't think quite honestly I mean I I certainly don't have any really clever ideas about it yet and we you know I don't know this week there's gonna be a visitor here who's uh who was a from Israel a guy yuram borak who's collaborates with h with Edvard um Moser and probably my Brit as well and and he's a former student of uh of um smolinsky I can't remember smolinski's first name who's really one of the grandfathers of computational Neuroscience you know I'm a lot of papers are coming through my mind now trying to sort of figure out how a tractor net works and how all these things end up representing I mean that's really what they're trying to get at but but they're really working on the mechanistic issues I mean how does it do the job that it does and they don't really unless until maybe until you liquor them up a bit they don't they don't want to they don't necessarily want to talk about the meaning question I mean how does this set of activities come to represent something in the way of meaning that thing you know now or how does it result in kind of experience and I mean we don't even have to like get into that but it's it's interesting that you bring that up because right I think you know we're kind of jumping around but we we got to this idea of it not being a map in the brain but for a while we sort of thought okay there's like a little picture in the brain and there's a topographic map and there's an image so um and through the now we still confuse all these words even thinking about computational representation in the brain is a similar I think a similar mistake to thinking of maps in the brain but it doesn't mean the word map or the word computation is like wrong it's just that we get like very confused you know about I don't think touches the problem at all so so you get so you get somebody like Tim Baron you know is arguing that some kind of a geometric sixfold representation is the common code for the way the brain represents information it's going beyond the you know go building Off the Grid cell notion you know that and how well the grid cell population tiles the whole environment you know he's jumped he's taken that idea and sort of said well you can find this kind of six-fold representational structure to the to the population code again all over the brand he's arguing and I think I and I talked to John about this as well and John we also think now he's probably gone too far here but even if that were the case even and you think about the grid cells I mean what is it about the that particular computational structure that causes it to mean what it means you know or that we think it means so to speak that that it what it means is something about the structure of the space that the that the orth animal is moving around in totally great this is great because I I think too that he's gone too far so to speak and trying to be so specific about the sixfold so on and so forth um but what I like about his work in that Tolman ien bomb machine is exactly that he puts Tolman and ien bomb together because I mean we kind of skipped over like so thinking about maps and then I think in your lecture you go into sort of feature detection and um contrast and Hubbert and visel and all this and it's kind of moving more towards where we are now population coding right where you're looking at regularities and so on and so forth and I bring that up but then also the Tolman iin bomb like I would like to hear your perspective about like those not fit together you know I bomb's idea and well what's the idea what do you see as the core idea in the in the in the Tolman icon bound machine what what is it doing well I mean I I think I would need to think about that a little bit more but I'll tell you what I'm trying to get at here in terms of this conversation so toon is associ with the cognitive map for example or the idea that like when a rat is moving through its spatial location that's also a kind of knowledge so to speak and that we could map it I mean I'm I'm now giving him words he doesn't necessarily use but in the same way we draw these little homunculi kind of things we could probably do that with certain individuals or certain rats in terms of how they're moving through uh space and how certain parts of whatever is happening in the hippocamp um is happening right get way too confusing I'm not saying it's a sixfold this and that and that tomman didn't care about that I mean he was no he doesn't care was not into the brain Tolman Tolman was just talking about the cognitive structure but that moving through a space is a kind of knowledge right I just think that's an obvious kind of thing that we can say the knowledge that Tolman claimed was the knowledge the kind of uh what leads to what knowledge I mean what Tolman claimed was is that the animals form a kind of a mental map a map of the world something like a map of the world in their in their head something like parts of the brain being associated with regularities in the way we described with the topographic map but not so no hard and fixed as that but like that going if you pushed him he might have said that but he really didn't ever talk about the brain yeah but that same kind of idea right that as you develop of course you kind of associate certain regularities with certain sure whatever functions yeah that would be the Modern Way talking about it so that's the Tolman part of it Tolman talked repres about the representations that our brains must be forming and the fact that they can be used as predict in predictive mode that you know good way to say it yeah and that basically they tell the animal if I do this then following is gonna happen you know you know he he connected all of those dots cognitively right which is which is great because that's kind of like a map right because you use it in order to sort of infer where you're that's why and I built on that I mean exactly and that goes into the beautiful book which we've discussed hippoc starting but so but what but so so now tell me about the I can bound part of it okay well this is what I don't think I think I'm missing something because I feel like there's a lot of controversy or fighting about something I don't know that I really understand iin bomb totally with this idea of Association um but I guess what I what I think is kind of clear and what I like about the Tolman iin bomb is that we can understand we could probably find some way I don't think anyone's found it but to understand that the same way that we study how for example we move through physical spaces um and that all of this that we've been talking about there are regularities and we could probably find maps and I mean that external maps that we draw or computational models that we make is what I mean by representation that we could also do that with other spaces like because we're setting what a space is you set the little room for the r but you could also set a little knowledge space so to speak and for me it's still all regularities that we've learned and you're still moving through a space in a similar way I don't mean that like the way we move through a room and the way we move through a book are going to look the same but there's probably some kind of pattern or regularity uh towards how we could understand those processes that is the same that's how I see that connection being made in a nice way but I could be wrong I mean I don't understand what iin bomb is so controversial about well I think controversy comes from the fact that you know this idea is built on the kind of I can bow me in Co I can Balman Cohen really notion of of relational memory you know and the and and the brain is coding these relations you know and the hippocampus in particular is a relational memory device so that would be this either or thing right so that then precludes all the other stuff we were talking about is that kind well not really because it because in I mean they may think it does that I don't think it does but seems like people take it that way and and then I can understand the controversy I guess I think the it's it's I don't think the idea is so much as so much controversial as vacuous okay what what is the idea you that because I think I'm missing what that Isa the vacuous part is that is that iin bound talked about this as a as as one of as one piece of the kind of brain's function this relational device right you know the hip campus is is specialized to kind of create these relations you know and it does it for space and for all these other things as well right that's the point you're making the problem is everything the brain does not just the hippocampus everything the brain does is relational there is no nonrelational part of the brain it does not exist it's it's a nonsense idea every neuron gets inputs and is relating inputs from multiple sources there is nothing that is non-relational it simply is not a it's not a useful explanatory idea so to the in my view and also John and I are very strongly in agreement on this there's there's just no it doesn't help you everything is Rel so that he's saying that so is he saying the hippoc campus's main purpose is to relate yeah his yes uniquely so okay and there other other parts of the brain don't have this kind of special you know ability to relate things to each other is the is the is the core of the I Bal cone relational idea and that is built into this Tolman ien bound approach you know that that you have this unique relational device but that's crazy the whole brain is a relational device everything the brain does is relation there's no okay see that I didn't get at all that it had to be a device that's relational to me it was much more of a theoretical um very much grounded in kind of things you start with in your book with space and how we how we understand space and as you as you said yourself they're talking about it as an association machine the Tolman ion bound machine is a machine that makes associations but but that's an association is just another word for a relation it's and the brain is in the business of of everything is associative there used to be in Psychology this kind of big distinction between associative and nonassociative learning as if there could be such a thing as nonassociative learning right non they still talked about as if as if associative I mean people don't talk the way I I agree with what you're saying but this confusing because when in all brain research we don't talk about the brain as being like that yeah no that's the reason why there's this controversy that some people are kind of taken up by this idea oh that's a really cool idea y you got this special relational assoc associative device the told I can bound machine blah blah blah without actually kind of digging deeper and you know trying to say well okay what's the rest of the brain doing is it doing non-associative things what does that mean every part of the brain is doing associative things there's no there is no distinction otherwise you know otherwise the brain would just be a bunch of isolated wires you know going from input to Output there'd be nothing else going on it's I agree but we don't we talk about associative memory as if it's one kind of memory we don't talk about all memory what I'm it what I'm trying to tell you is that so-called non-associative memory which were things like habituation sensitization they're not non-associative I agree 100% but it's not the way you're taught that you know so I can just imagine a lot of people aren't even understanding no no you're absolutely that's what he's saying you know because absolutely right that the objection I'm making is not one that is widely kind of moted in the you know in the in the literature at this point but sooner or later people going to realize that it's a vacuous that this idea I think they're going to realize that this is a vacuous idea it doesn't get you anywhere Bas I don't think it's vacuous but I think it's not um like an answer it's a for me it's like a tool to rethink some of these things like even you know toling I love toling and these different kinds of memories but we do need to do some rethinking about how we categorize all all these memories and things and I guess like being a little bit kind of you know not really under I haven't studied I have studied uh iin bomb and all this from an angle that I from which I want to see things right not and everyone's doing that so I can see how this gets incredibly confusing but to just not try to stick it to to to to like a theory I think what's interesting is like for me and you might really disagree with this is like that doesn't seem so far from a cognitive map in the Tolman sense of like um like I wouldn't say for example that okay we're just just because we can understand that uh cognition is um that when we're moving through a space that is also a kind of knowledge then we just say okay everything is movement and everything is knowledge I mean we could also kind of do that too right and that would be vacuous um but instead if you really look at at what's being said there and you start to understand um there's actually very big huge differences in terms of these parts of the brain um or even if not right even if some other part of the brain could become something like the hippocampus it would still be interesting that the brain itself some in some way yeah is developing some kind of regularity that can then be used in many different situations to yeah I believe however that every part of the brain is doing what you're just saying that that is not unique to any particular system that's just the way the brain works it every part of the brain is extracting is engaged in extracting the regularities from the input patterns that it receives basically and and those input patterns necessarily include relational information they have to they always do they're not about like one point in space they're always about things that that involve relations so my problem with that approach is that it doesn't seem to me to add any useful explanatory power to the to any other story that one you know that one it's you know it's basically gets back to everything is assoc you know everything is associative and you know that again the philosophers were okay with that idea you know that the brain works on the basis of associations that's that's what it does and it and it's not this part of the brain that works on the basis of associations and extraction of statistical regularities and so on it's every part of the brain is doing exactly that that's kind of incredible I mean I think if you take I we don't have time to go into all of your work but the things we've talked about in other yeah discussions together and the reason that I believe this whole kind of toll on through you through the Mosers all the stuff that you're now celebrating there I think at the in Tron time I think this is incredibly important because it does show us something that does tell us something about what we were trying to talk about about about um how we can be in the world and experiencing the world and that's like a continuous thing does absolutely it does but we're back to the kind Inside Out versus outside in and try why does it have to be either or I mean this is you're right and and what is the what is the magic sauce that is going to help us understand you know how both of them both the internal structure like the that shows up at day 10 you know in dependent of experience and the actual experience that the organism has which actually then refines the shape of the Taurus and refines and then the grid cells emerge after the Taurus maybe it is something like the association though and I don't mean that in a vacuous way but I mean connected to all the research you guys are doing you do start to think of the brain in that way yeah yeah it is and you think of it in that immediate sense you brought up at the beginning actually we get somewhere really new and beautiful I hope you're right but do but the problem is to go back to representations is we can't get rid of that trying to find it in the brain or find it in the world right instead of understanding Association as a process that can't be isn't represented in the brain or the world but we can represent with the books we write and the computational models we make and the images we draw I mean the notion of a representation in the brain may just be a slice through through a moment in time and that the representation is really some combination of the structure of the brain and the structure of the of the world and the inputs from the world to the brain and the and the tight coupling between those two and and the and the kind of compromise between those either or positions is figuring out how that coupling Works basically and and when you say you know that this activity in the brain represents x x about the world then something about the world is you know part of that story it has help create that you know that that set of brain activities either phenetically or ontogenetically and you know the answer is going to be found somehow in the interactive kind of re-entrant you know recursive sort of language that the brain uses you know to sort of interact with the world and that's kind of what we're working on and trying to figure this out is the the the middle the middle ground has to be you know somewhere between those two and and how does it and how does a how does a particular Network come to have the meaning that it has we're no longer talking about patches or Maps but you know an entire network or population of of cells how do they come to mean what they mean and and and how do they come to have the power they have to to generate Behavior so to speak and it's got to be some interaction between what's out there in the world and the experience the organism has and the and the kind of the pre-wired structure of the system which was pre-wired because of millions of years of uh of evolution you know that built it that way because it actually worked in the world and so it's got to be connected to the world in that way not a free device basically the point I was trying to make about to and IAM was that no matter how you think about this it's going to be an associative device of some kind right and so yeah yeah but that's that might be the contribution that I'm thinking he's bring because I don't think we talk about it like that and and and I don't think it's actually vacuous to talk about it like that if you if you bracket it in a in the way we're talking about not if you just say okay the hippocamp is this little device in the brain that does this okay that's and it does it through this and this and this okay then we're getting like that's not gonna help us I'm open to that possibility but I see I do see a way that those things aren't there could be help helping each other this tradition that it's you know so important to me that you guys are all there are doing and what he was trying to say sometimes in probably almost a cartoonish sense about about there being a way in which um this is happening in all these different contexts you know that's the and how does that and how does that work but you know what what Believe It or Not we've we've run out of time I know I know we have to me I have to I have to quit now so we can yeah so we can pick this up again you know next month or you know we can set up another talk back in Tucson yeah will you um can I read your your talk that you're going to give there maybe uh I will this history thing yeah or or or send me some of the stuff you were just talking about the hybrid hybrid stuff too uh that that's not ready that's not ready well then maybe your talk that you or something talk which is really a different thing is just interesting history uh I can send you the once I have the PowerPoint finished then I can send you this then I can you know and I think a lot of it on the slides I mean some of it some of it won't be but most most of the story will be on the slides okay that would be great that yeah okay let's do that done with it I won't be done with it for another week or so but I'll when I'm done with it I can send it to you and I'm in I'm in research mode so I'm gonna be I'm finally staying still so when you get back and settled just let me know we can I'm back the second of October so we can set something up for the middle of October yeah I'm around in OCT now okay that would be perfect yep sounds good have fun there yeah always fun send me the paper