How the Brain Creates Multiple Models of Reality | Inner Cosmos

[00:00:00]what is special about the wrinkly outer layer of the brain the cortex and what does this have to do with the way that you come to explore and understand the world and by the way why do you see a whole image when you open your eyes even though each part of your visual cortex has access to only a tiny bit of the image and for that matter the brain is divided into different areas for Sight and Sound and touch and so on and so why when you're ping a cat why does the cat seem unified why doesn't the sight of the cat seem separate from the purring and the feel of the fur can we build a new model of how the brain works and in what ways is what the brain doing something very different than what's happening in current AI welcome to Inner Cosmos with me David Eagleman I'm a neuroscientist at Stanford and in these episodes we sail deeply into our three-b universe to understand why and how our lives look the way they [Laughter] do today's episode is about a new model of the brain developed by my friend and colleague Jeff Hawkins and we'll get into an interview with him shortly but let me preface by saying that for centuries people have stared at the brain and tried to figure out how this thing works because when you stare at it it's just a huge lump of cells you can see that there's a wrinkled layer on the outside and when people dissect it they can see that that part is about 3 mm thick and it looks a little different it looks grayer and so that part is called the gray matter and we call this the cortex which means bark like tree bark and the stuff below that thin layer is called white matter and it looks white because the tiny data cables coming off the cells the axons these are wrapped in a little sheath called myelin which makes it look white okay now what you immediately notice by looking at brains across different mammals is that all the stuff you find under the cortex all the subcortical stuff looks essentially the same horses and elephants and mice they all have the same architecture going on that we do they all have a Thalamus and hippocampus and cerebellum and so on but there's one thing that really distinguishes us from our cousins and for that we return to the gray matter the cortex it's not that our cousins don't have a cortex what distinguishes US is the absolute enormity of our cortex we humans have a ton of this stuff so take four pieces of paper from your printer and place them next to each other to make one really large piece that's how much cortex a human has if you were to spread out the wrinkles now our nearest Cousins the great apes only have about one piece of paper worth and most mammals have a lot less than that so something about the story of the runaway human success has to do with the fact that we have way more cortex for our body size than any other creature and side note I'm really talking about what's called the neocortex or new cortex because we also have a little bit of paleo cortex or old cortex but the thing that really makes us outstanding is the amount of neocortex that we have but what is this neocortex doing well if you look at any Neuroscience textbook you'll see that this part of the brain the cortex is often drawn with different colored regions like this red region over here is devoted to vision and this green one is devoted to hearing and this yellow one to touch and so on but something I've been obsessed with and WR about in my latest book live wired is that this is the wrong way to think about it because the neocortex is remarkably flexible it's not a fixed map if you are born blind the part of your cortex that we would have thought of as visual cortex gets taken over by hearing and touch and so on now let me just be really clear clear what I mean by taking over the neurons there are the same the cortex looks exactly the same from the outside but the function of those particular neurons is now not visual they have nothing to do with visual information anymore now that same neuron instead of firing when it detects a moving object now it responds to a touch on your toe or hearing a B flat note or whatever so the little labels that we draw onto the brain these maps that we impose these are actually massively flexible and as you may know I gave a talk at Ted about this a while ago where I showed that you can feed in new kinds of information let's say through the ears or the skin and the Brain will figure out how to deal with that data it will flexibly devote part of its cortical real estate to that and this line of thinking LED some scientists like Vernon mount castle some decades ago to realize that the cells of the cortex are a one-trick pony no neuron is inherently a visual neuron or a neuron devoted to hearing or touch or smell or taste or memory or whatever all parts of the cortex are perfectly capable and willing to take on any job so that suggests they're all running some sort of basic algorithm and it doesn't matter what kind of data you feed in different parts of the cortex will say cool I'll build a representation of that data I don't care if it comes from photons or air compression waves or temperature or whatever I'm on the job here to build an understanding of whatever is coming in locally now it's not individual neurons that are building models but instead groups of many tens of thousands of neurons arranged in a six layered cylinder so think about this like you're a geologist and you drilled out a cylinder of rock and you saw six layers in it six sedimentary layers that's what the neocortex looks like six layers and it's built out of these columns which have the same types of neurons with the same connection patterns in each column and so think about the cortex as being made of lots of these columns like taking hundreds of thousands of grains of rice and standing them up on their end and attacking them all next to each other now people have known about cortical columns for many decades since Vernon Mount Castle first discovered these in 1957 but recently someone has pulled together several different threads to propose how this could underly what the cortex is all about and that's someone is Jeff Hawkins and his team and so I met with Jeff in my studio now Jeff is one of my favorite people because he does theoretical neur Neuroscience he really tries to figure out the big picture of what the brain is doing now Jeff has a very interesting history so I'll just mention that in the 1980s he was a graduate student at Berkeley where he proposed a PhD thesis on a new theory of the CeX but his proposal was rejected and so he ended up pursuing his vision for mobile Computing instead and in 1992 he launched the company Palm which made the Palm Pilot if you remember remember that this was this little handheld device and you could write on it with a stylus and it would translate your handwriting into text and you could use this for your address book and your calendar and your contacts and note taking this was the first entrance into the world of portable Computing and it really changed the world anyhow a decade later Jeff returned to his original love which was theoretical Neuroscience trying to figure out what's going on with the brain and he wrote a book in 2004 called on intelligence which was very influential on me and lots of other thinkers I know so I was very excited when Jeff recently came out with his next book that represents his last decade and a half of research it's called a thousand brains a new theory of intelligence and it describes his framework for thinking about the brain so without further Ado let's dive into a very cool new model of the brain [Music] okay Jeff so you are a theoretician you think about the brain from a high level we're in this era now of AI where AI is doing all kinds of things that are amazing and no unexpected but you see the brain as being very different from what is going on with let's say large language models so tell us about that that's absolutely true um you know the current AI wave is really amazing but those models don't work at all like the brain and I think you could start with one really fundamental difference um brains work through Movement we move our bodies through the world we move our hands over objects to touch and learn what they are we move our eyes constantly so the inputs of the brain are constantly changing but mostly because we're moving through the world and the term for that is a sensory motor system and the Brain can't understand its inputs unless it knows how it's moving through the world so we learn by exploring by moving different places picking things up touching them so on and that's all animals that move in the world learn this way so this idea that the brain is a sensory motor system has been known back in the late 1800s but it's pretty much ignored by everybody yeah that's right but it leads to a very fundamental different way of how we acquire knowledge and how knowledge is represented in the brain whereas um today's AI is most of it's built on um well deep learning or Transformer Technologies which essentially feed data to it we don't it doesn't explore and we feed uh to large language models we just feed the language so there's no inherent knowledge about what these words mean only what these means words mean in the context of other words right but you and I can pick up a cat and touch it and feel it and know it's warmth and we understand how his body is moved because no one has to tell us that we just experience it directly um so this a this is a huge gap between brains pretty much all brains work by sensory motor learning and almost all AI doesn't and and you can just peel the layers of and see what the differences are and it makes a huge difference um so I'm not a fan I'm a fan of AI today but I don't think it's the future of of AI I don't think it's going to get you to what people really want are truly intelligent machines um okay terrific and we'll dive into that more in a little bit now when we look at let's say the human brain there's lots of areas that we can point to there's the cortex the rinkle outer bit there's all these subcortical areas um when you think about intelligence and uh and the stuff that we're going to talk about today what is the part that you concentrate on well we we concentrate first and foremost on the neuro cortex which is about 75% of the volume of your brain I mean it's what you see as you said if you could take a scope and that's what you see the Neo cortex um and so it's a pretty dominant part of what we think of intelligence you can't consider it completely on its own I mean it's connected to all these other things and so we also study those other things and in service to the the neocortex so we studied the thalmus and we study the cerebellum and we studied the basa just because you have to know how the cortex um works with these other things but our primarily our goal and and many neci this goal is to understand the new yor cortex because that's what mammals have we've got a big one you know everything we think most of what we think about being intelligent about our ability to understand the world and generate language and see and hear and so on is the New York cortex not 100% um but most of it and fortunately Al not only is it the biggest structure but it's a very very regular structure so you can look at this thing the nework cor is like a sheet of cells it's you know it's like a size of a large dinner napkin and only a few millimeters thick and it gets wrinkly because you stuff it in your head um that everywhere you look on it it looks remarkably complicated and remarkably the same so the areas are doing Vision look like the areas are doing language look like the areas you're doing Touch look like the areas are doing everything really and so there been long speculated that there's sort of a common uh algorithmic principle that's applied to everything everything we do all of our sensory inputs all of our thinking all the language it's hard to believe but the evidence is overwhelming and so our research has really been to understand what is that that algorithm the cortical algorithm often referred to as a cortical column um you know this repeated structure that seems to underline vision and hearing and touch and thought and everything we do um and that's that's just just an appealing thing to try to understand and we've cracked it we've actually we actually cracked it we understand what's going on that's awesome okay so uh a couple of things right so the way I sometimes phrase this to people is that if I had a magical microscope and could show you a part of the brain and you could see all the activity running around in the cortex there could you tell me is that visual cortex or auditory or somata sensory and the answer is you couldn't tell me and I couldn't tell you because it all looks the same through and there's and as you know there's these experiments people done where well first of all if you have trauma to one part of the cortex other parts will pick up the same function um you can also people rerouted sensory inputs to different parts of the cortex and animals and they steam the work so for example you have uh visual information instead of going to the back of the brain the visual cortex that gets rerouted to the auditory cortex and that auditory cortex becomes visual visual cortex right it's incredibly powerful and flexible system um and mammals you know all we have we have a sensors um quite a few actually more than most people think because a skin is a lot of different sensors but um other animals have different sensors and they have cortex too and so um there seems to be this Universal algorithm that is can be applied and now we know it's a centry motor algorithm um uh it can be applied and we and we've spent decades trying to figure this out and we've cracked it oh that's awesome just before you tell us about that so tell us what a cortical column is okay so imagine we talked about the new yor cortex is sheet of cells like 3 mm thick um a cortical column is a little section of that going through the three 3 mm it can vary anywhere from like a a third of a millimeter to a millimeter in diameter it's it's it's a it's not something you would see it's not like it's sitting there to be plucked out but we know they exist um and so within that let's let's say it's a 3 millim tall and a half millimeter wide cylinder that goes across the the cortex that is contains all the neural Machinery that you would see anywhere in the cortex and each cortical column um because they look like a little grain of rice in some sense why you can imagine lots of little grains of I stack next to each other um each cortical column gets input from some well in parts of the B they get input some patch of sensory input so from patch of the retina from patch of the the CIA patch of your skin other parts get information from parts of the the your cortex so cortex is connected to Cortex but each one is looking at a small if you think about the primary sensory regions of the C T which are quite large um they're getting input from a small sensory area right and so people used to think that well if if this little column is only getting input from a small part of the Reena it can't really doing very much right it can't be very smart all it could do is process a little piece of the information there and therefore maybe it's going to detect an edge or something like that and there's a lot of evidence for that but we now know what's happens is that um the the cortical columns they get input from over time from different parts of the world so eyes are moving like three times a second and so that cortical C is be looking at three different things every second and it can integrate how the sensor is moving how your eyes are moving with what it's sensing to build models that are much larger than it can sense in the same way that you could take your finger in a in a dark room and say okay David I want you to learn this new object let's call it you know I don't know a coffee cup you've never touched and so what you could do is you touch the coffee cup and you move your finger long and around and as you do you build a threedimensional model of the cup even though you're only getting input from one fingertip the eyes are doing the same thing it's surprising you don't realize this so every cortical C what we understand now is doing this sort of processing movement information and sensor information building what we call structured or 3D models of things in the world so it's quite different than even most neuroscientists think about it uh and there's a lot of reasons we can talk about how it was missed uh for all these years so in the CeX you have essentially six layers of cells and a column is um all six layers it's all six layers it's going up and down uh it's like think of it like layers of a cake column is you're taking a a straw and shoving it through the top and so you've got this call got a straw of cake okay great and so the idea is if you're looking at some column in um you know in primary visual cortex your point Jeff was that um you know it's it's like looking at the world through a straw it only sees a little tiny piece of the world but because the eyes are moving out because you're exploring the world this is actually getting lots of parts of information it's exploring the world in the same way that your fingertip explor and it has to integrate information over time that's the key right and you can literally do this you can look at the world through a straw right and and you can say oh what am I looking at well you can't tell until you start moving the straw and then you can start and you can also learn objects that way so literally you can learn by looking through a straw which is what sort of what one column is doing got it and in your model there are thousands of such columns and each one of these is learning a model of the world as it's going so tell us about right right so um I think this idea that there's all these columns is not a new idea um and that they had this fundamental algorithm but but we were I think the first people to kind of figure out what what it is and what it's doing so the trick of this thing is it's it's a little tricky here you know when you look out at the world you have a sense anybody you have a sense where things are I have a sense where you are relative to me I have a sense where this microphone is Rel to me I know where my hand is all this cop I now there turns out that do you have any kind of sense of location in space you have to have neurons representing it there's nothing goes on in the brain if there aren't neurons firing doing it turns out most of the Machinery in the newor cortex is keeping track of where things are relative to other things so those six layers all those cells at least half of that circuitry is tracking where the sensory input is coming from in the world so if I move my finger over this coffee cup the part that's getting information from my the sensory like I'm sensing an edge for example as I move my finger it has to keep track of where my finger is a location of it and its orientation Rel to this cup is quite complicated um but that's what it has to do to build this models and now we know how it does it there's all this evidence for it um so the brain is just trying to keep track of or all of its inputs are in the world all relative other things then it builds up these threedimensional models of the world so tell us about how it does that then right so you can think about when you're in high school you learned about cartisian coordinates XYZ coordinates right and so if I want to say where is something where are you relative to me I might say okay your nose is the origin and I could say it's some distance from here and you know X Y and Z well you have to have something like that and um but brains don't do it that way they do it another way and and um this was some very clever research in the last 20 years that people discovered in the anal cortex and the hippocampus these cells called grid cells and play cells which actually operate as reference frames they are they they are a way of neurons to represent locations and they work differently than xlin so there's no origin it's kind of really clever how they work um Nature has discovered a different way of doing this so tell yeah make sure you tell us a little bit about that well okay but these these are wellknown things let take grid cells which andronic Forex what they do is they these cells if you take a set of them individual cells could are not unique any individual cell may say I fire at different locations in space but if you take a cell of them they're unique and so you can encode a unique location in space and the key thing about them is these cells automatically update as you move so the original grid cells are where your body is in a room and um as you move it's called path integration it says okay you're moving at this this direction at this speed so we'll just automatically update these neurons as if we know where you are right and so it's h it's what sales used to do a dead reckoning you would just say oh I I you know I could I'm heading north for an hour at three knots therefore I'll be three miles ahead in this direction so we know that these cells exist um they've been well studied people when Nobel Prize for these things so we speculated that the same neural mechanisms these grid cells um and equivalents would be in the cortex in every cortical column and sure enough they're finding that now so there's all kinds of res now they're finding in humans and other animals that there are grid cell like structures in cortical columns and so what does that tell you it tells me that that's the mechanism by which the brain uses for reference frames and so literally when you build a model of something in the world like a model of a cup or a model of anything it's essentially what you're doing is you're saying here's the sensation and here its location here's another sensation a different location here's another sensation a different location you add all these together and you get a threedimensional model you can say this thing consists of these features in these locations relative to each other other um and so literally in our head we build models of the world that are three-dimensional analoges of the physical things we we interact with um and that's why you appear three-dimensional to me you know you're not an image you're a threedimensional structure because I have a threedimensional model of humans and I have a special model for you David okay great and so what okay so you've got these columns in the cortex they building threedimensional models or keeping track of where your fingertips are where your eyes are so we've got these different Windows into the brain you've got these data cables coming in carrying spikes it's all spikes but some of them carrying visual information some auditory some touch um brain doesn't know that by the way exactly right spikes or spikes it's all spikes exactly right and so for any particular column it might only be getting a subset of those tells about that right right well any well I'm not Shing me a subset well what I mean is uh if I if I am cortical column that happens to be sitting in visual cortex and I happen to be getting visual information but I'm not getting auditor so there's a real one of the first things we had to address with this series is why does the world appear unified right I don't feel like you know I I don't feel like oh I'm touching something with my hands and I'm looking at something else with my eyes it's all one thing there's this cup right and I feel the warmth of it and I know I it's one thing it's and yet we have all these different models so it turns out you have models of Cups and that are taxle models are based on how you how it feels you have models of how it looks um you might even have a model how it sounds like this particular ceramic cup I have expectation what it would sound like I put on this counter here might different on a ceramic counter um and and yet these models are they're all independent but they're not completely independent so there's these long range Connections in the cortex that go from all different sides the left side of the brain the right side of the brain all over the place just lots of different types what they're essentially doing is they're voting they're all saying like one my finger says I think I'm touching something that feels like a cup I I may not be certain another thing I have something too that's I'm not really certain in the eyes thing and they very quickly reaching set the only that makes sense for all our input is we're all looking at the same object and so there's like a across these long range connections it'll solidify into a a percept that's what you perceive you don't actually normally perceive the individual Sensations from your eye or your fingers you just say I'm holding this cup in my hand and it's one percept and so it's these long range connections and how these columns vote all the time this is why I can Flash an image in front of your eye and um and say okay well each column is looking at a part of that image who decides what the whole image is right and and by the way I don't even have time to move my eyes once I've learned objects I don't have to move my eyes to recognize them what we call a flash inference um the reason is because each part of the CeX visual cortex it has hypothesis about what it might be saying and they they vote and the only thing that makes sense is the final thing they agree upon so I have to learn by moving my eyes by attending to different things and mov my fingers but I don't always have to to infer or recognize things by movement I don't always have to I can just flash an image in front of you and you see I know what that is and you don't have the time to move your eyes this fooled a lot of vision researchers for many years because they assumed that movement wasn't necessary because I can Flash an image in front of you but you can't learn that way you have to learn by attending to different things quite right just so it's clear uh to the audience so this issue about voting um it's not that they're all their votes to some Central agency it's that they're all talking with one another simultaneously simultaneously and something about the spike patterns pulls into shape right right well we know exactly how this occurs Le we have models of it and we we've simulated and it matches of Neuroscience um it's a little it takes a little while for people to get the sense of it you're right there's no Central voting tally it's like it's and I don't have all the comms don't have to talk to all the other comms it turns out they only have to talk to a few other comms as long as everyone talks to somebody and the whole thing is connected see it doesn't have to like gazillion connections um but it's it's more like you have a neural you know imagine neurons are spiking and um and let's say I have let's say I have 5,000 neurons that representing what I'm seeing that's not that many actually so 5,000 neurons and in the brain we're getting a little technical here um activations are typically sparse meaning of those 5,000 cells maybe only 2% or 100 are active at any point in time the others are silent um and so I'm representing something by U saying there's 100 neurons active out of 5,000 now if I wasn't certain I might say oh well let's do this I'm going to say it could be object a it could be object B it could be object C and I'm GNA activate them all the same time so now I have 300 neurons out of 500 they're simultaneously active now that might seem confusing but it isn't NE have no trouble with this and and everybody's doing the same thing they're all getting multiple hypothesis and it's very quickly says the you're supporting this hypothesis and you're supporting this hypothesis it happens simultaneously no one has to go through Surly there's no like counting the votes so let's try this hypoth and this it all settles very very quickly um it's kind of cool thing have you thought about what happens when you settle on a hypothesis and then you switch for example looking at the neckar cube this cube made out of 12 lines what you know you see it one way then you see it the other way what is it that allows it to switch all right now imag A necro cube is a two-dimensional image right it's a two-dimensional image of a threedimensional wireframe cube or something like that right um and so it's not three-dimensional it's really two-dimensional but your brain wants to make it three-dimensional right because it doesn't know two-dimensional things that look like that and so it everything we try to do fits into our models right right we don't say oh that's a two dimensional let me just can't be a cube no you oh no that's got to be a cube because I know cubes I don't think it looks like that's not a cube so wants to settle on hypothesis of like okay well this corner is in front of that corner and this corner is behind that corner this corner to the left of that corner it just has to do that to fit its models that's right but why doesn't it land on a hypothesis and stick there well I don't really know but there's other people hypothesize about this is that the evidence goes both ways right there's multiple hypothesis and so neurons have a way of getting tired about what they're doing after a while they say you know literally they have they have a way up they say you know I'm not going to keep find like this forever you know things are changing the world we don't just get stuck so there's there's various speculated mechanisms for how neurons and it's been observed will sort of you know say okay I'll be active for a little while then I'm G to stop and let someone else try something right got it but what it means is that the other hypothesis has to be kept alive somewhere somehow well it it may be not it may be just like I have this hypothesis I locked in on it and now I'm going to say that's no longer possible just go back to square one what is possible h you know so it's not like I have these two images in my head I conceptual or perceptually you don't feel that way right you do only one or the other so you lock in the one the other is forgotten um but then if I say disabled the first hypothesis we're not going to allow that be anymore then it say okay what's possible oh this one's possible I'll switch to that one um it's not like they're both active one's active and then it gets tired and then the other one I work so coming back to the main thing one part that I want to return to you is just this issue um that a particular column might only be receiving touch information another column might be receiving only auditory information and so on well they build independent models right they I can I have a tacle mod of an object I have a visual mod of an object right they're not the same the visual model of the object will have color perhaps the TCT will have temperature and uh texture and things like that um so they're different models but because they can vote you have a single percept of it yeah okay and one the things that's important here which of course you emphas you and I both emphasize this a lot in our books is that all we are ever seeing is our model of the world right and so we we don't have any direct access to what's actually out there and so the fact um you mentioned earlier The Binding problem I don't know if you mentioned it by name but the binding problem is this issue that when the coffee cup is here and it's moving how come the color doesn't bleed off the cup and how come it seems like one thing and so on buying problem is is a poorly defined problem exact it means a lot of different things to a lot of different people so you got to be really careful say oh I let's talk about The Binding problem um uh I might have a different perception of what the binding problem is to me The Binding problem is the one I've already discussed which is you have these different uh sensory inputs that uh but somehow they lead to a single percept and you can switch back and forth it's like I don't it's like how do I bring these things together how do I say these are all the same thing and the people used to think in The Binding problem is like oh if if I have the auditory cortex and the visual cortex and sematic sensory cortex touch then they must all project to some place where they are bound together into a single model and we flip that on its head they don't bind they bind together through just long range connections but there's no central place they have to do that there's no nobody sitting on top of it and saying hey what's your vote what's your vote no it's just like so there we don't need a model that incorporates all the aspects of of objects we have independent models that we can invoke as needed and they all they all vote to reach this common consensus so I have no problems navigate you know doing things in the dark I have no problems doing things just by Vision I have no I can do things sometimes with audition um yeah like I know the same things are going on I have the same model of the world right you know if I if I'm walking at night between my bed and the bathroom and it's pitch black I still have the same model of the house I I still know where the door is going to be and everything else you know but I can do a touch versus Vision right so there isn't a cenal model that says here's the model of my house of touch and vision hearing it's all these independent models right and now the reason you called your book a thousand brains we call this hypothesis the Thousand brains Theory theory is um is precisely because you've got all these cortical columns and they're each making a little model of the world and they're all talking to one another right so you know hi this is H this feels like coffee cup this looks like coffee cup this sounds like coffee cup when I place down this is the temperature of coffee cup and so um and so these are all talking with one another so so the reason I call the Thousand brains is that a each cortical column is doing what the entire brain is doing right each Coral column is a sensory modal Learning System um and and when we ask where is a model of something we've been talking about this where is a model of the skull or the microphone or whatever the so many things we know where is that model it's not it's it's in many different places so there's a thousand models of coffee cops is a thousand models you don't perceive that but they exist and um and so it's like it's it was really trying to capture that original idea that cortical columns are common and that and that there's all these different models out there that are different and they can vote them on you know they vote to reach a consensus yeah um and that's certainly consistent with the idea that you know for example if someone is born blind and the visual cortex gets taken over by hearing and touch so on they are better at hearing and touch because they just have a lot more real estate devoted towards right right more real estate and a lot more practice too right um right so uh it's amazing how flexible it is yes given your model of the brain let's talk about Ai and what you think is going on currently with llms and what that is missing right llm flip is interesting well let me start with the criticism of AI in general okay AI has always been focused on what they call benchmarks like how well can you solve problem so how well can this system recognize images how well it can play chess how well it can play go how well it can translate from one language to the other and you have all these benchmarks and everyone competes against these benchmarks that are kind of diverse all over the place that's the wrong way to think about it um when we let's use computers as an analogy when we say something is a computer we don't base on what it's doing we base on how it works Allan Turing and joh for Norman defined what we now call a universal turing machine which is like okay if a system has memory and a and a processor and the memory has data and instructions and you can change the instructions and change the data it can do anything and that is a computer so I can say my toaster is a computer even though it's a very limited computer because it has one of those things inside right there's if it was a hardcoded with springs and wires and stuff it wouldn't be a computer but because it has a little microprocessor has those definitions it's a computer so that's how we do it in the computer world we say these are the these are the functions that has to form and you can apply to big problems little problems different types of problems all over the place in AI we've been focused on this idea that oh benchmarks you know and we always want to be beat some human well like a dog almost everyone who has a dog says it's intelligent right but doesn't have language it doesn't play Shad it doesn't play go but why do we say it's intelligent because we can tell that that dog has an internal model of the world it's kind of like my internal model he knows where the door is and knows how to get a going on a walk and so why why focus on this issue of like well it's not intelligent because it doesn't play go better than the best human player so um I think part of the problem was that people didn't know how brains worked and so if you don't what are you going to do right well we don't know it we are we know enough to build this stuff so I think in the future that's what's going to be we're going to say AI systems don't have to be like humans they don't even have to do the same things humans do some of them are going to be very dedicated there very focused task some are going to be very broad some might be you know Engineers building space stations all this huge variety but they're all going to work on the same principles that biology has discovered um today AI doesn't work on those principles you know most of it if we talk about the large language models these are transformable models we feed in a string of tokens basically words or word like things and it just learns the structure of that string and it's very good at what it does but has no inherent knowledge the actual word the world it doesn't have a three-dimensional model of the world it doesn't if if someone's written about it it'll tell you about it but it can't experience it itself so you couldn't send one of these AI system down to space and say you know go to Mars explore and see what's out there that we can build things with and hear some tools and start building a structure there's no way they're going to do this stuff just not going to happen but the cont the tools we're working on can do that that's what humans do um and that's the promise of AI is not just you know targeting things that humans can do like high level things like you know translating language or writing poems or things like that it's really how do you build a system that understands the world and knows how to act in that world and that's the key yeah so given that one of the things you wrote in your book that I thought was great was uh you addressed this issue of the existential threat of AI that a lot of people are banging on about and you don't think it's a threat I don't think it's a threat I mean you have to you have to tease it apart because so many people like there's different exential threats but you know the one is called the alignment problem like oh these AI agents are going to you know you're going to tell what to do but it won't be aligned with our values and I'm just saying they don't have any values and and the it's just so far from reality I just if once you understand how brains work is they not going to do any of that stuff it's it's hard it's hard to me to give a a scct answer to this but I don't think that today's AI systems have any of these problems um they're not going to run away they're not they not gonna have their own desires um they're not going to say hey I'm awake I need to survive you know it's just be because these current large language models are just statistical parrots that are taking a bunch language and spinning language back out right and you can apply no app to Robotics and other things but they're going to be still sort of statistical par exactly but and they might by the way they lack human brain we talked earlier the neor is the biggest part of the brain but we have a lot of other parts of the brain and our emotional centers and how much what makes us humans our drives and motivations are mostly not the ne cortex right they're these other things and if you provided an AI system with those other things I might start worrying about it but but if you're just trying to model stuff it's it's not a threat um uh it's we just assume that some AI system because it can spew back language is going to think like us and be like us and have our same motivations nothing like it at all so tell us about the Thousand brains project and how you're going to make this happen right so we we kind of been working on this theory for decades really and and maybe five or six years ago we really had some breakthroughs and sort of all came together and then we said well I always thought that this is the way we're going to build truly intelligent machines and this is at the same time as deep learning and Transformers taken off and all this excitement about it um but that didn't distract us um we said okay let's see if we can start building this stuff so we for a couple years we had a small team that was trying to implement the Thousand brains Theory uh modeling Coral columns the voting all this stuff multisensory things all this stuff we we're modeling it and and we decided earlier this year that the best way to go forward would to do this in an open source project um we hadn't actually told people we were doing this before so we've created the Thousand brains project um we're taking all of our um our code and putting in open source we are taking the patents we have a lot of patents we're going to make a non assert Clause we've done that non assert clause on our patents we have um we hired a team of people to like um open source project manager for outside people we've got already got quite a few people interested um we s we already have uh received some funding from The Gates Foundation for this significant funding to help fund the project for a couple years we have there's a guy named John Shan At Con melon University who's building silicon to implement cortical columns so so there the people around the world who've been excited about our work been following it and want to join in on this thing so we figured let's get them all together let's build a a framework open source project um and uh so we built out this team uh we have um it's run by the woman Vivian clay who's just brilliant and technical side is by D Neils um and um and so we H we're just starting this you know so we we actually haven't officially and we talked about it but we haven't officially launched it yet because not everything is open yet we have to there's a lot of stuff you have to do to put in to get those all to work uh but we're we're going full more in this and I think my hope is that um anyone who's excited about the working there quite a few people um can help join us and work on this and Propel it forward and really created um what I not only just an alternate form of AI sensory motor AI based on brain principles but I think what's going to be actually the ultimate U primary source of AI um which is brain modeling um I a thousand brains project this is amazing so how do people get involved in this uh you can just go to our website Thea and and.com and you'll there's a lot of information there already a tons of information uh it's like we have all the stuff we've accumulated documentation code uh videos again all that up there plus tutorials and so on so you just you can just uh you can you go to our n.com and it'll be obviously how to sign up uh to be informed or what's going on or how to get involved great so if some listener to this podcast says I want to get involved and understand more about they go to n.com and they can they can first of all they'll start they'll sign up to getting notified about things are happening they can get educated on the whole project they can um I don't think right yet they can contribute code yet but that will happen within a month or so it'll be obvious how to get started there's there's a lot of information to learn I would think if you haven't if you haven't uh you might want to start with just by reading the B of thousand brains uh because it gives you the not only the basics of the theory but it also gives you the vision about how this is going to play out over time and so the idea is just so I'm straight on this so the idea is a person can download the code and run this model they can run right the um first we're we're making so that you can do that you can run our current experiments okay um you can recreate them you can apply them different ways great so something that you and I have in common that we are obsessed about is this idea that we're living inside our own internal models this is all a construction and you had a line in the book that I loved which is that if you had different sensors for picking up different information in the world we would have a different perceptual experience like a completely different experience of the universe well maybe completely not not completely like a blind person is learning the wor of touch and a a person who is death and a person maybe has sensory problems on his hand they will end up with a similar structure world sorry but what I mean is not in terms of hi we pick up on the visible light range but I pick up on infrared and you pick up on radio waves okay right you might if you if you really did that then you would have a different view of the world like it's often you know like you take the issue of color it's often said that be is you know see in the ultraviolet and we don't so what looks like to us is a white flower to them is this beautifully colorful variegated flower right but let's say you saw in a totally different part of the in electromagnetic spectrum and so you see in the microwave range question is would would we have color at all I don't know it's it's hard to say right there's a there's an underlying a really interesting philosophical problem called qualia which is like why does color feel like color right and it doesn't feel like sounds or tactile Sensations um and it's an interesting challenge uh to understand that I've written about it a bit um yeah do you have a hypothesis about this I'll tell you what mine is but it's tell me yours it's only sort of half one which is I think it's about the structure of the data coming in defines the qualia now I don't know why or how that's true but you know with the eyes you got two two-dimensional sheets of data coming in and so Vision feels like something with hearing it's a one-dimensional signal it's just going up and down and vibrating your eard drum that feels like something you don't confuse Vision with hearing those like completely different worlds to you my interest has been in what happens when we feed new structures do you've done a lot of interesting stuff in this area exactly would you have a completely new quality is it possible I think so I mean um certainly you can imagine first of all I agree with you um again it's all spikes right so there's nothing there's no color spikes there's no heat spikes it's just spikes and so obviously the the different quality has to come about somehow from the structure of the data spatially and temporarily um and also sensory motary you know it's like how things change as you move and I think that's a big part of it so I agree with on a fundamental that it's it has to be somewh in the data and not it's nothing else um and um and we can then ask ourselves something like well imagine you've been blind your whole life you don't have a sense of color you've never experienced color and so to you would be kind of mysterious thing someone can say well can't you tell that's that's you know that's a this type of orange and that type of what are you talking about right they have to accept that you have some super sense and the world looks different to you because you have vision and I don't and they may be able to touch things that you know sense things that I don't sense like if you ever tried to read Braille if you're not a braille reader feel like what St it's a blur right so they oh no I feel everything there right so we can we can just ask ourselves the questions like what's the world like to different people and sometimes we'll end up with a similar model like yeah well you and I would have no matter what stances you have we'd have the model of physical structure of a coffee cup but um but other times it could be quite different you know and if you start like sensing parts of the the radio your spectrum or um other things it just be you know one of the thing I always wondered like what would be like if you had if you had smell sensors under your fingers right um and then Everything You Touch well we kind of we have we have temperature sensors and we have t all kinds of but what if you smell like you could tell chemicals that were on the surface of object this is what dogs do you know dogs they don't just smell they stick their nose right on the thing and they smell they move to the next spot they smell dogs build this threedimensional Str structure of smells we don't have that smells for us just kind of like wasting in from some direction right dogs have this incredible model of the world smell model and it's hard to imagine what it is but I'm sure they have it so I think it's fun to think about these things um um I don't you know in the future we'll build machines that perceive the world different than we do but that'll be great yeah okay Jeff this has been wonderful thank you for being here today thanks David it's always great talking to you I enjoy it it's a lot of fun we're we're and I love your your podcast [Music] though that was Jeff Hawkins theoretician and author of a thousand brains now I love his model because it builds on previous research and gives us a possible starting point for how this whole system might be working this is a view of the brain in which you don't have just a single model of the world being constructed but hundreds of thousands of little models each viewing the world through their little straw and these models are independent but they're not completely independent so they communicate with each other and they vote and in this way the whole system converges on its best guess of what's going on out there in the world and by this mechanism we construct a full three-dimensional representation of the environment around us with its sights and sounds and three-dimensional structure so this gives us a clear framework for thinking about the neocortex now we might not know for a while if this answers everything or it needs some tweaking or if there are far better models coming down the pike but what I absolutely love about this is that this is where the Endeavor of science shines taking something that seems insanely complex 86 billion neurons with 200 trillion connections something of such vast complexity that it Bank rups our language and saying wait what if there's a really simple principle at work here what if there's a way that we could reduce all that complexity by just looking at this from a new angle so let me give an analogy here just think about what it would be like if you had a magical microscope with which you could look into a cell and into the nucleus in the middle what you would see is mindboggling complexity there you'd see millions or billions of M molecules racing around and interacting and doing God knows what and you'd say wow there's no way we're ever going to understand this but then Crick and Watson come along and say actually the important thing is this DNA molecule and keeping the order of these base pairs and all the rest is housekeeping and suddenly the fog of confusion lifts now something that seemed well beyond us can be described in a sentence or two and science leaps forward and things move fast from there I worked with Francis Crick when I was in my post-doctoral years and now I look around me at Stanford and Silicon Valley and there are thousands of Laboratories and companies doing amazing work with genomes and their existence results entirely from this one simplifying insight about DNA in 1953 that new model that suddenly clarified what is happening inside the nucleus by the same token this is what we're trying to do with the brain brains appear to be ferociously complex and yet we have lots of brains running around the planet we've got 8.2 billion of them so something must be straightforward about their architecture or else Mother Nature wouldn't be able to build these over and over with such reliability you couldn't drop this massive quantity into the world and have them all functioning well unless there was something pretty uncomplicated about building and running a brain so that is the overarching game of science to take the overwhelming complexity around us and to find new angles to look at things to reveal simplicity go to eagan.mn with questions or discussion and check out and subscribe to Inner Cosmos on YouTube for videos of each episode and to leave comments until next time I'm David Eagleman and this is inner Cosmos [Laughter]