How the Brain Constructs Reality: Illusions, Memory, and Consciousness (NOVA)

[00:00:01]♪ ♪ ♪ ♪ ANIL SETH: The brain is one of the most complex objects that we know of in the universe.

[00:00:13]BOBBY KASTHURI: There are more connections in your brain than there are stars in the Milky Way galaxy.

[00:00:20]So, we literally walk around with about 10,000 galaxies' worth of neuronal connections in one of our brains.

[00:00:27]HEATHER BERLIN: That vast web of connections creates you.

[00:00:29]But how?

[00:00:30]NANCY KANWISHER: Figuring out how the brain implements the mind is a massive challenge.

[00:00:37]♪ ♪ SETH: It seems as though the world just pours itself into the mind through the transparent windows of the eyes and the ears and, and all our other senses.

[00:00:49]♪ ♪ BERLIN: But is what we see, hear, and feel real?

[00:00:57]You might think that the reality outside is actually what you're perceiving.

[00:01:02]And the answer is no, it really isn't.

[00:01:05]Almost at the very first moment, we are transforming reality.

[00:01:09]It feels so real because we don't know better.

[00:01:12]Think about illusions.

[00:01:14]ROSA LAFER-SOUSA: Do you remember the dress?

[00:01:15]Of course-- it's like a celebrity, the dress.

[00:01:18]A polarizing debate that took over the internet.

[00:01:22]White and gold.

[00:01:22]Blue and black.

[00:01:23]SETH: Illusions are fascinating; they're like fractures in the matrix.

[00:01:27]BERLIN: Ow!

[00:01:28]Isn't that interesting?

[00:01:28]Whoa!

[00:01:30]They reveal to us that the way we perceive things isn't necessarily the way they are.

[00:01:35]BERLIN: Could you be the biggest illusion of all?

[00:01:39]SUSANA MARTINEZ-CONDE: Your sense of who you are is an illusion as everything else; you're no exception.

[00:01:46]BERLIN: "Your Brain: Perception Deception."

[00:01:49]Right now, on "NOVA."

[00:01:51]♪ ♪ MAN: Okay, rolling.

[00:02:05]♪ ♪ Take three.

[00:02:32]BERLIN: Have you ever thought about what's real?

[00:02:35](echoing) Somehow the whole world out there gets inside my head.

[00:02:46]How do I know what I see, what I hear, what I feel is right?

[00:02:53]It's a question that's fascinated me ever since I was a little girl.

[00:02:58]I couldn't sleep one night, and I had this thought for the first time: And then I thought, well, even if I don't have a body, can I at least keep my own inner thoughts?

[00:03:14]So I asked my dad the next day, "Dad, where do my thoughts come from?"

[00:03:19]And he said, "They come from your brain."

[00:03:23](explosion echoes) ♪ ♪ "Your brain." I was hooked.

[00:03:30]This bag of jelly between my ears, how does it work?

[00:03:34]STANISLAS DEHAENE: I think it's one of the ultimate mysteries.

[00:03:36]How matter becomes thought.

[00:03:40]ANDRÉ FENTON: To answer that question would be perhaps the highest human achievement to date.

[00:03:46]DANIELA SCHILLER: I mean, forget about scientific quest.

[00:03:47]It's a human quest.

[00:04:03]♪ ♪ BERLIN: To find answers, I became a neuroscientist and a psychologist.

[00:04:16]I'm Heather Berlin, and my journey to understand my brain begins with a question.

[00:04:23]How does the world out there, with all its beauty and complexity, get inside our heads?

[00:04:30]♪ ♪ Think about it.

[00:04:35]Imagine for a second you're a brain, sealed inside your skull.

[00:04:39]There's no light, no sound.

[00:04:44]KASTHURI: You're a massive collection of billions and billions of cells that are living in this weird pond that is entirely devoid of all of the sensations, and that somehow, through chemistry and electricity, all of these perceptions and memories of the world originate in our brains.

[00:05:03]BERLIN: All brains-- from the tiny fish to the enormous elephant-- contain microscopic cells called neurons, and one of their jobs is to translate input from the external world, whether that's light, heat, sound, or pressure, for instance, into electrochemical signals the organism can use to act.

[00:05:21]What might be surprising to you is that as neurons process sensory signals, they create an edited version of reality, even on the most basic level.

[00:05:32]KASTHURI: We're deciding to throw away 99% of the world.

[00:05:35]Almost at the very first moment, we are transforming reality into something we could use.

[00:05:40]BERLIN: Neurons transform reality by competing with each other.

[00:05:44]When a creature touches, smells, sees, or hears something, its sensory neurons fire; some a little, some a lot, depending on where the physical signal is strongest.

[00:05:54]But follow those signals down towards its brain, you'll see that the weaker ones get stamped out.

[00:06:01]For simple brains, say, the brain of a crab, a diffuse light to the eye becomes a sharp beam.

[00:06:07]For more complex brains like ours, it's in part what makes you think that these two squares are completely different colors, but actually, they're identical.

[00:06:18]♪ ♪ MARTINEZ-CONDE: Think about illusions.

[00:06:26]First, they're a lot of fun, but as neuroscientists... Whoa... MARTINEZ-CONDE:...illusions are very important to us.

[00:06:33]Because of this discrepancy between objective reality and subjective perception, we can use these illusions as a handle to try to understand what the brain is doing all the time.

[00:06:48]BERLIN: Susana Martinez-Conde, along with her partner and collaborator Stephen Macknik, are among the world's preeminent experts on illusions and perception, and what they tell us about how the brain works.

[00:07:00]Ha, now what?

[00:07:02](both laughing) MARTINEZ-CONDE: To give a different example, Adelson's checkerboard illusion, this is so striking because you see some of the checks as dark and others as bright, but you realize that it is exactly the same shade of gray.

[00:07:18]BERLIN: Don't believe it?

[00:07:20]Look at the squares labeled A and B.

[00:07:23]A looks darker, right?

[00:07:26]Wrong-- that's the illusion.

[00:07:29]That's because your brain is adjusting for the shadow.

[00:07:32]MARTINEZ-CONDE: What's happening is that your brain is considering the light source and basically subtracting that light source from your resulting perception.

[00:07:41]Your brain is performing an interpretation, a shortcut, if you will, to arrive at a perception.

[00:07:49]BERLIN: If the brain's shortcuts distort reality this much, how much of the world are we really seeing?

[00:07:56]STEPHEN MACKNIK: What you need to understand is that we really can't see most of the world around us.

[00:08:02]We're effectively blind to 99.9% of the world around us at any given time.

[00:08:06]If you hold out your thumb at arm's length... Mm-hmm.

[00:08:09]And you straighten your elbow and you look at your thumbnail, your thumbnail is about one degree of visual angle here, and it turns out that that's the only place we can actually see with 20/20 vision.

[00:08:19]Wow.

[00:08:19]And everywhere else, we're legally blind.

[00:08:22]BERLIN: It might sound hard to believe, but human vision is really like this.

[00:08:27]You actually only see detail in about one percent of your visual field.

[00:08:31]That's because only a tiny portion of the world can be processed in detail by the retina.

[00:08:37]It feels like I'm seeing the whole world in 20/20 vision.

[00:08:40]And it's almost all completely made up in your brain, based on assumptions and models of how the world works and just a tiny bit of high-quality visual information.

[00:08:50]Let me demonstrate this to you.

[00:08:51]I know it's kind of hard to believe... Mm-hmm.

[00:08:53]...because you've been having your whole life where you feel like everything's continuous.

[00:08:55]Yeah, show me the data.

[00:08:56](laughs) Show me the evidence.

[00:08:58]Let's look at an eye-tracker and look at your eyes and how they actually work.

[00:09:01]And if you put your head in this headrest... Mm-hmm.

[00:09:04]...we'll point the camera at your eyeballs and we'll actually be able to see where your eyeballs point during this demonstration.

[00:09:11]Feels like "Clockwork Orange." (chuckles) "Buck lived at a big house in the sun-kissed Santa Clara Valley."

[00:09:20]BERLIN: First up, a reading demo.

[00:09:22]Though most of your screen may be filled with Xs, to me, it just feels like normal reading.

[00:09:27]I barely see the Xs, and that's because the display of letters is tied to my eye movements.

[00:09:33]BERLIN: Well, it's just, the words are being revealed depending on where I look.

[00:09:37]That's right, so as you move your eyes... So weird.

[00:09:39]...the words are revealed to you.

[00:09:41]But we don't move our eyes in the same way you do.

[00:09:43]So we just see a bunch of Xs most of the time.

[00:09:46]BERLIN: What turns out to be critical is my eye movements.

[00:09:49]MACKNIK: So our eye movements program what part of this high-quality piece of visual real estate we're going to put where and at what time.

[00:09:57]BERLIN: The human eye moves about three times per second.

[00:10:00]We take it for granted, but without these movements, we'd be basically blind, as Steve is about to show me.

[00:10:07]MACKNIK: In this demonstration, it's the opposite.

[00:10:09]Here we're blocking what you can possibly see, right?

[00:10:12]BERLIN: Though you may see a whole scene with a square moving around, all I see is the square!

[00:10:17]I can tell something is around the edges, but it's blurry.

[00:10:20]Whenever I try to look, the square moves with my eyes and it's blocked.

[00:10:25]BERLIN: This is so frustrating, this one.

[00:10:27]Who has their hand up in this image?

[00:10:32]BERLIN: Uh... I think that guy down there?

[00:10:35]MACKNIK: That's right, but it's very hard for you to see, right?

[00:10:37]Every time I look at him, it, yeah.

[00:10:39]It disappears because this block, it blocks it.

[00:10:43]MACKNIK: This actually is interesting because it's in high-quality vision wherever you look, but it's blurry in the surround.

[00:10:51]BERLIN: Now the scene looks normal to me, but mostly blurry to you because your eye movements don't match mine.

[00:10:57]MACKNIK: Wherever you happen to look, you have high-quality image processing happening and the surround is completely blurry.

[00:11:05]This kind of represents exactly what your visual system looks like all the time anyway.

[00:11:11]So why would our brains be built this way?

[00:11:14]Well, think about what the alternative is.

[00:11:16]What if we didn't have eye movements?

[00:11:17]Well, if we didn't have eye movements, and we just wanted to see the entire world, we'd need to have our retinas see everything in very high quality.

[00:11:23]Our brains would be 600 times bigger, and you gotta remember, the visual system's our best sense.

[00:11:28]This is our richest sense.

[00:11:30]So our other senses are, are even more impoverished.

[00:11:33]BERLIN: Here's how your brain really sees the world.

[00:11:36]It's easy to think it's like this.

[00:11:38]You open your eyes and the whole world pours in.

[00:11:41]But really, it's like this.

[00:11:45]Your eyes sample tiny pieces of the world and the brain fills in the rest-- constantly, all the time.

[00:11:52]KANWISHER: We feel like we have this incredibly rich, wide, full, detailed percept of what's going on moment to moment, and that's probably pretty illusory.

[00:12:01]What we're actually aware of is a tiny subset of the information that comes in through our eyes.

[00:12:06]BERLIN: Don't believe it?

[00:12:08]Consider this: your optic nerve is what connects your eye to your brain, and its location near the center of your retina effectively creates a blind spot near the center of your visual field.

[00:12:19]And yet, you don't experience the blind spot-- why?

[00:12:23]The brain samples the area near the blind spot and fills in the gap with its best guess.

[00:12:29]KASTHURI: It's probably not fair to say that we completely confabulate the world, it's just that we probably represent one percent of it at any particular moment in time.

[00:12:38]So, it's a constant updating between what I see with, versus what I remember, versus what I expect.

[00:12:44]And it's that dance between those three that actually gives us our sense of reality.

[00:12:49]BERLIN: And amazingly, that edited reality-- despite its limitations-- serves us quite well.

[00:12:56]KASTHURI: You might ask, "If I'm just keeping track "of one percent of the information in the world, how can I drive a car?"

[00:13:02]And it turns out that first one percent of the information that comes in from the world is actually an enormous amount of information. (chuckling) If we had to actually pay attention to everything on the road at the, at one particular time, it would take minutes, maybe even longer, before I decide to turn the wheel right or to turn the wheel left.

[00:13:21]♪ ♪ BERLIN: By understanding how my senses really work, I'm getting a peek behind the curtain: what my brain is really up to outside of my awareness.

[00:13:33]MARTINEZ-CONDE: Based on this very tiny amount of information, we construct this grand simulation of the visual world around us.

[00:13:44]It feels so real because we don't know better.

[00:13:49]BERLIN: And most of the time, we all agree on that simulation.

[00:13:52]It's when we don't that we can learn something.

[00:13:56]So do you remember the dress?

[00:13:56]Of course.

[00:13:58]Did you see this dress or this one?

[00:14:02]It's a simple question, but the answer has divided friends and family.

[00:14:06]White and gold.

[00:14:06]Blue and black.

[00:14:08]I remember it caused quite the stir, right?

[00:14:10]Massive stir.

[00:14:11]A polarizing debate that took over the internet.

[00:14:14]♪ ♪ LAFER-SOUSA: People had existential crises over this image.

[00:14:20]People tweeted things like, "If that's not white and gold, my life has been a lie."

[00:14:24]Swear on your mother's grave.

[00:14:26]Because that dress is white and gold.

[00:14:28]Out of her (bleep) mind.

[00:14:29]LAFER-SOUSA: Massive arguments.

[00:14:31]I watched videos of people screaming at each other.

[00:14:33]GRAYSON DOLAN: This is white, dude!

[00:14:35]ETHAN DOLAN: White? That is dark blue!

[00:14:37]It's purple-blue!

[00:14:38]LAFER-SOUSA: I bet there was a divorce here or there over this image.

[00:14:43]BERLIN: So when you first saw that dress, as a, as a vision scientist, what did you think?

[00:14:47]Well, when I first saw the dress, I thought it was blue and black.

[00:14:52]And I thought that the internet was yanking my chain.

[00:14:54]Right-- to get the goat of vision neuroscientists.

[00:14:57]Sure.

[00:14:57](chuckles) But in the morning, when I looked at my phone, I saw white and gold.

[00:15:03]And now, of course, I was obsessed.

[00:15:05]So I said, "Well, if this is an ambiguous image, all I have to do is disambiguate it."

[00:15:09]So I set to work, I got into Photoshop, cut out the dress, put it into a scene with lots of rich cues.

[00:15:16]BERLIN: Hm.

[00:15:17]LAFER-SOUSA: And all of a sudden, boom: you can see the dress is white and gold.

[00:15:20]Wow.

[00:15:20]Now, the pixels, the pixels that make up the dress there, are identical to the original image.

[00:15:24]Okay.

[00:15:26]BERLIN: Now, this doesn't work for everybody, but for most, the visual context can make all the difference.

[00:15:33]LAFER-SOUSA: What's different here is, her skin is tinted blue, the background has blue light cast on it, she's standing in the shadow of that cube, and so your brain says, "Aha, I need to ignore "some amount of blue light "that is in this signal that's hitting my eye and render this as white and gold."

[00:15:51]BERLIN: And if we flip things around?

[00:15:54]LAFER-SOUSA: Same dress, pasted it into this other scene.

[00:15:58]Her skin is tinted yellow, the background has a yellow cast, she's standing no longer in the shadow but in the light.

[00:16:04]Boom! Blue and black.

[00:16:05]Amazing, that's really amazing.

[00:16:07]So again, the dress, the pixels are exactly the same.

[00:16:09]LAFER-SOUSA: Identical.

[00:16:11]BERLIN: The dress is a powerful example of how color really works in the brain.

[00:16:15]Does that mean we're creating color in our mind, or does color actually exist in the world?

[00:16:21]Color takes place in the brain, and I've prepared a little illusion for you that should convince you of this.

[00:16:27]LAFER-SOUSA: So I have a picture of four cars here.

[00:16:30]I want you to tell me, what color are these cars?

[00:16:33]Let's start on the top left.

[00:16:33]BERLIN: Okay, so the one on the left looks red, then the one next to it looks blue.

[00:16:38]I'd say the one, the bottom there, bottom left looks green, and then the one next to it looks orange.

[00:16:44]LAFER-SOUSA: Okay.

[00:16:44]Mm-hmm.

[00:16:46]What if I told you that all of those pixels are not only gray, they are the same gray?

[00:16:52]BERLIN: How is this possible?

[00:16:54]It's because the light that enters your eyes, contrary to what you might have learned in school, is not color.

[00:17:01]Color is an interpretation of your brain.

[00:17:04]Here's how it works.

[00:17:06]Light shines on the world and bounces off objects-- this part you know.

[00:17:11]And light comes in different wavelengths, each corresponding to a different color.

[00:17:16]What you might not have heard in school is how those wavelengths change when they hit different surfaces-- rough, smooth, wet, et cetera.

[00:17:24]This signal that gets into your eye is actually a product of the reflective properties of the object and the wavelength of light hitting it.

[00:17:33]Then that signal is focused on the retina, the back of the eye, where we have about 130 million light-sensitive cells.

[00:17:40]Three types called cones are involved in color, each sensitive to different wavelengths of light: long, medium, and short.

[00:17:48]But that light still isn't color.

[00:17:51]For that to happen, our brain has to take that three-piece code from the retina and use the relative response of the cones to encode color.

[00:17:59]It's not until that signal gets to an area called V4 that we get a neural representation of color that corresponds to our perceptual experience.

[00:18:09]So why would our brains be built this way?

[00:18:12]Well, if our brains weren't built this way, objects would appear to change in color all the time, and that would render color a pretty useless signal in the world.

[00:18:24]♪ ♪ BERLIN: That's because objects reflect different wavelengths into your eye depending on the lighting conditions.

[00:18:32]If your brain didn't compensate for this, a red berry would appear gray in a cave, blue at dawn, and orange at dusk.

[00:18:39]But instead, your brain carefully calibrates your experience to hold color constant.

[00:18:45]Similarly, color vision in other animals is tuned to their needs.

[00:18:50]SETH: Different species have very different kinds of color vision that are suited to their particular environments and their particular challenges for staying alive.

[00:19:00]BERLIN: Dogs rely on smell, so they have fewer types of cones, and thus see the world like this.

[00:19:06]Birds need to recognize tiny color differences from great distances.

[00:19:10]so they have an extra type of cone that allows them to see more colors than we do.

[00:19:14]And bees need to find flowers rich in nectar, so they see ultraviolet light that's invisible to us.

[00:19:21]LAFER-SOUSA: Color provides a lot of valuable information about the world, but only if we can faithfully extract something about the object.

[00:19:32]So we don't actually see color as it is in the real world, we just see it in terms of how it's useful for us?

[00:19:39]Absolutely.

[00:19:40]And the dress is probably the best example of that.

[00:19:43]It's a really powerful demonstration of how our color machinery works.

[00:19:48]BERLIN: So why do people see this image of the dress differently?

[00:19:52]It comes down to your brain's assumptions about the lighting conditions.

[00:19:57]It seems that the more time you spend working indoors under artificial light, which is predominantly yellow, the more likely you are to say the dress is black and blue, because your brain assumes it is lit by artificial light and subtracts out the yellow.

[00:20:11]Conversely, if you spend more time in natural light, which is bluer, you are more likely to see it as white and gold.

[00:20:18]So then what is the actual color of the dress?

[00:20:22]Well, Heather, I happen to have brought it with me.

[00:20:27](chuckles): Wow.

[00:20:27]So what color is it?

[00:20:29]Uh, it's obviously, I was right, blue and black.

[00:20:32]Team blue and black for the win.

[00:20:32]Yes, yes.

[00:20:34]I can't believe this is the actual dress.

[00:20:36]BERLIN: I feel like I'm holding, like... It's like a celebrity, the dress.

[00:20:40]I know, it should be in a museum, not in my closet.

[00:20:40]Yes!

[00:20:42](both laugh) Before the dress, people hadn't really realized we differ so much between individuals.

[00:20:48]We're now quite used to the idea that we all differ on the outside.

[00:20:52]We all have differences in skin color, in height, in shape.

[00:20:55]But just as we all differ on the outside, we all differ on the inside, too.

[00:21:00]And this inner diversity is very important.

[00:21:03]It gives us a certain humility about our own ways of seeing.

[00:21:06]BERLIN: Illusions give us a ringside seat to watch how the brain creates our world.

[00:21:12]And it's not just the visual domain.

[00:21:14]Try listening to this.

[00:21:17](staticky computerized voice playing) Brainstorm, right?

[00:21:20]Simple enough.

[00:21:21]Now, listen to this.

[00:21:23](staticky computerized voice playing) Green needle.

[00:21:25]Okay, so you're thinking, "What's the big deal?"

[00:21:28]But, what if I tell you that the two audio clips I just played were exactly identical?

[00:21:34]For most people, what you hear depends on which label you read.

[00:21:38]Sounds unbelievable?

[00:21:39]Here, try it again, but this time, just read one.

[00:21:44](staticky computerized voice playing) Okay, now read the other and listen again.

[00:21:49](staticky computerized voice playing) Now, when I first encountered this, I was floored, too.

[00:21:55]Even though I know what's going on.

[00:21:57]When your brain encounters uncertainty, it fills in the gaps with its best guess.

[00:22:02]In this case, we have a degraded audio clip, and when you're primed with a certain word to go with it, your brain automatically jumps to the best fit.

[00:22:11]For most of us, we literally hear what we want to hear.

[00:22:14]And it gets even worse-- let's try one more.

[00:22:18]Another internet sensation that lit up debates across the country.

[00:22:23](cleaner computerized voice playing) Once and for all, is it yanny, is it laurel?

[00:22:30]It's not yanny, it's laurel.

[00:22:30]It's yanny!

[00:22:33]Did you hear yanny?

[00:22:35](cheering and applauding) Who heard laurel?

[00:22:38](cheering more loudly) It is laurel and not yanny.

[00:22:42](remix of computerized voice playing) It's like that stupid dress again all over, but in audio form!

[00:22:48]This is not saying "laurel," this is only saying "yanny."

[00:22:51]Exactly, it's laurel!

[00:22:54]Now, about half of you hear yanny, and the other laurel, and unlike the first illusion, I can't get most of you to experience this one any other way.

[00:23:05]You're locked into your version of reality.

[00:23:08]Experts aren't exactly sure why, but some of us seem to pay more attention to the low frequencies, laurel, and others to the high, yanny.

[00:23:17]The divide stems from the fact that the audio file is an ambiguous signal made up of both high and low frequencies.

[00:23:24]But by manipulating the frequencies, I might be able to change what you hear.

[00:23:29](computerized voice playing at mid frequency) High... (voice plays at high frequency) (voice plays at low frequency) Low.

[00:23:38]All of this goes to show how much the brain is an active interpreter of sensory input.

[00:23:43]Our perception of the external world is actually much less objective than we'd like to believe.

[00:23:49]Most of the world around us is very real, but you just never lived there, okay?

[00:23:54]You lived in your mind, which is a perception of that world that's being filtered through a bunch of salt water sacks of proteins and electrochemical signals, which can't possibly be making completely accurate determinations of what's actually in the outside world.

[00:24:13]Not convinced? Or maybe you're just asking, "Why?"

[00:24:16]Well, try watching for when the green dot flashes.

[00:24:21]Does it line up with the red dot?

[00:24:23]If you are like most people, the red one always seems just a little bit ahead.

[00:24:27]Now try again.

[00:24:32]The red dot and the green dot are actually perfectly aligned.

[00:24:36]That's because some neuroscientists would say that it's not your brain's job to perceive the world accurately.

[00:24:42]Rather, its job is to predict what happens next.

[00:24:45]To a certain extent, you see what you expect to see: a predicted path of motion.

[00:24:51]And this is what helps us hit a home run or flinch from a punch at just the right moment.

[00:25:00]The brain is a predicting machine.

[00:25:02]Given a set of circumstances in this story at this moment, what are the likely plausible next events in the story?

[00:25:15]SETH: The brain is using sensory information to calibrate, update, to fine-tune these predictions so they remain tied to reality in ways that are not constrained by accuracy, but that are constrained by how useful the brain's perceptual predictions are in the business of staying alive.

[00:25:41]BERLIN: And to keep us alive, the brain has evolved to look for signals of potential danger.

[00:25:46]One of the most important is pain, and as neuroscientist Theanne Griffith is about to show me, sometimes that can be a kind of illusion, as well.

[00:25:54]BERLIN: So what is this?

[00:25:56]GRIFFITH: This is a thermal grill.

[00:25:56]Okay.

[00:26:00]This is a machine that could give us some insight as to how pain works in your brain.

[00:26:05]All right, this is making me nervous already as I'm getting strapped in!

[00:26:07](Griffith laughs) GRIFFITH: Don't worry, it's all an illusion, actually.

[00:26:11]Okay.

[00:26:13]And it's comprised of these different metal bars that are either set to a cold or warm temperature.

[00:26:18]So why don't you go ahead and touch that first bar?

[00:26:21]It's warm, right?

[00:26:23]And then the next bar?

[00:26:25]Cold.

[00:26:25]Mm-hmm.

[00:26:26]GRIFFITH: And then the next one, warm.

[00:26:28]You see? So they're alternating cold, warm, cold, warm.

[00:26:28]BERLIN: Mm-hmm.

[00:26:30]Now, you want to see what happens when you put your hand down?

[00:26:33]Not necessarily.

[00:26:33](both laugh) Go ahead.

[00:26:35]Okay.

[00:26:37]Okay, here we go.

[00:26:39]BERLIN: Ow!

[00:26:40]GRIFFITH: Right? Isn't that interesting?

[00:26:40]Whoa!

[00:26:42]Yeah, what is going on there?

[00:26:44]It sort of feels cold at first, but then... Then it gets this kind of burning sensation, right?

[00:26:48]Yes, very much so.

[00:26:49]BERLIN: It feels super-hot, like I'm getting burnt.

[00:26:52]So it's not 100% clear exactly how this is happening.

[00:26:56]But what we think might be going on is that, basically, your brain is getting a little bit confused.

[00:27:02]Okay.

[00:27:02]It's feeling cold, and it's also feeling warmth.

[00:27:05]And somehow, it's interpreting these two signals as pain.

[00:27:09]BERLIN: Here's what neuroscientists think is going on.

[00:27:13]In your hands, you have separate sensors for heat, cold, and pain.

[00:27:18]Normally, when you touch something slightly cold, both your cold and pain sensors are activated, but the cold ones override the signals from the pain sensors, telling your brain there's nothing to worry about.

[00:27:30]Unless, in this very unnatural scenario with the thermal grill, you happen to be touching something warm at the same time.

[00:27:37]Here, the heat signals cancel out the cold ones, leaving you with just the pain ones activated, telling your brain, "Ouch!"

[00:27:47]So in that respect, is pain real?

[00:27:49]Mm-hmm.

[00:27:49]Or is it just an illusion or a construct of the brain?

[00:27:52]That's a really good question.

[00:27:54]So noxious stimuli is, is a real thing, right?

[00:27:57]If you stick your hand in boiling water, that's an aversive stimulus.

[00:28:01]The perception of a noxious stimuli is real.

[00:28:01]Mm-hmm.

[00:28:06]Pain is more of a construct, right?

[00:28:06]Mm-hmm.

[00:28:08]And it can vary from individual to individual.

[00:28:10]EMERY BROWN: Pain is a construct of the brain.

[00:28:14]How do we know that?

[00:28:16]You touch a needle, right? And prick your finger.

[00:28:19]We can draw the anatomy of what just happened.

[00:28:21]We have very well-defined pathways saying, "This is pain information."

[00:28:27]We don't interpret it as pain until it hits your brain.

[00:28:33]BERLIN: Pain, not unlike the experience of color, is a construct of the mind.

[00:28:37]Mama! Mama!

[00:28:39]BERLIN: But just because pain is in your brain doesn't make it any less critical for survival.

[00:28:44]GRIFFITH: Pain is a very important... (gasps)...learning mechanism for children.

[00:28:49]They learn what behaviors they can engage in that are safe and what behaviors, well, they should not engage in because they could cause them bodily harm.

[00:28:57]And there's, um, uh, different mutations that people can have in certain proteins that make them completely insensitive to pain.

[00:29:04]And so kids do things like bite on their lips or on their fingers when they're very young, and as they get older, can engage in risky behavior.

[00:29:12]So pain is extremely important for us to feel.

[00:29:16]SETH: Illusions are fascinating.

[00:29:20]They're like fractures in the matrix.

[00:29:22]They reveal to us that the way we perceive things isn't necessarily the way they are.

[00:29:28]MACKNIK: Illusions help us find the cracks in the mortar of that world we've built for ourselves, and understand what it is our world is actually made out of and what the brain is actually doing.

[00:29:39]So most people think of the brain reconstructing the world more or less verbatim.

[00:29:45](dog growling) But that's just not true.

[00:29:47]What it's actually doing is, it's getting very little information and it's using that very little information to make a big, grand model of the world.

[00:29:57]MARTINEZ-CONDE: We cannot process the vast amount of information that is constantly bombarding our senses.

[00:30:05]Illusions, you can think of them as shortcuts.

[00:30:08]Shortcuts make us faster, more efficient with less resources.

[00:30:15]Based on these snippets of information, we build this more complex simulation of reality.

[00:30:23]And that simulation of the world is what we call consciousness.

[00:30:29]BERLIN: Consciousness.

[00:30:31](alarm buzzing) We take it for granted, but every time you wake up, (alarm stops) your brain stitches together all your sensory inputs-- the sound of a distant train... (train whistle blowing)...the smell of coffee, the warmth of the sun-- into an experience of the world.

[00:30:50]And that experience, that awareness of the world, is what scientists call consciousness.

[00:30:57]In neuroscience, consciousness is the Holy Grail.

[00:31:00]Humans have been fascinated by consciousness for thousands of years, probably much longer than that. (chuckling) Take three.

[00:31:08]Now, of course, the word "consciousness" means a lot of things to different people.

[00:31:12]To some, consciousness means being awake, as opposed to asleep.

[00:31:16]Or self-aware, or the contents of my thoughts.

[00:31:19]But that's not how we neuroscientists think about it.

[00:31:22]We think of it as something much more basic-- it's just internal experience.

[00:31:30]It feels like something to see the color red.

[00:31:33]To taste a strawberry.

[00:31:35](thunder rumbling) To hear the crack of thunder.

[00:31:40](thunder crashing) SETH: We are complicated biological creatures, but the most central feature of our lives is that we are conscious creatures, too.

[00:31:51]When I open my eyes, it's not just that my brain does some sophisticated processing of the visual information.

[00:31:57]I have an experience.

[00:32:00]BERLIN: During the course of my journey, I've seen how our experience of reality is not what it seems.

[00:32:05]If my conscious awareness is built from my perceptions, flawed as they may be, how does that work and what does it mean?

[00:32:15]♪ ♪ So please take a seat.

[00:32:20]BERLIN: Some of the first clues trickled in from people like this.

[00:32:23]LORELLA BATTELLI: Put your chin on the chin rest.

[00:32:25]BROWN: A lot of very valuable information comes from patients who've had, part of the brain's damaged.

[00:32:30]By piecing these various parts together, seeing what was lost, we've come to appreciate the role that these various brain regions play in the creation of consciousness.

[00:32:39]We're going to calibrate your eyes first.

[00:32:41]SETH: A powerful example of this is the phenomenon of blindsight.

[00:32:46]BATTELLI: This is a patient who had a stroke in her visual areas, in the back of the brain.

[00:32:53]And this stroke is affecting her visual field.

[00:32:55]BERLIN: Three years ago, she felt a pounding in her head.

[00:32:59]WOMAN: I had what I thought was a migraine.

[00:33:01]I actually went to the emergency room because I'm walking around with this area where I can't see.

[00:33:06]BERLIN: The stroke damaged a piece of the brain devoted to vision, leaving her with an apparent total blind spot.

[00:33:13]WOMAN: That blind spot, it's enough that if you're driving, an oncoming car disappears into it.

[00:33:20]It's a little anxiety-producing and, and things like that.

[00:33:24]BERLIN: In everyday tasks, her eye movements make up the difference.

[00:33:29]But what happens when she doesn't move her eyes?

[00:33:32]Neuroscientist Lorella Battelli wants to find out, so she developed a clever series of experiments to pin down, just how blind is she really in that spot?

[00:33:42]BATTELLI: We're using EyeLink, which is the eye-tracking system to make sure she doesn't move the eyes.

[00:33:47]BERLIN: She keeps her eyes focused on the center spot.

[00:33:50]Every time she hears a beep, she has to say if those little dots inside the circle are moving to the left or to the right.

[00:33:56]Left.

[00:33:58]BERLIN: The eye tracker checks that she's not shifting her gaze.

[00:34:03]WOMAN: Right.

[00:34:04]When you're doing tests like this, that blind area, what does that kind of look like for you?

[00:34:09]What does it feel like for you?

[00:34:11]When that target pops up in my blind area, I don't see it.

[00:34:17](machine beeps) Left.

[00:34:21]BERLIN: Strangely, even though she says she doesn't see anything in the blind spot, she gets it right more often than not.

[00:34:26]WOMAN: Right.

[00:34:28]BERLIN: So that some information is getting in.

[00:34:30]Yeah, so... But they're not consciously seeing it, but they can respond to it in different ways.

[00:34:33]Exactly.

[00:34:35]BATTELLI: Even if they say, "I didn't see anything."

[00:34:38]Left.

[00:34:38]But you tell them, "Please, just tell me whether you saw it or not," then their response would be above chance.

[00:34:44]BERLIN: So just keep your eyes closed, okay?

[00:34:47]And... BERLIN: What's going on?

[00:34:51]To probe deeper, Lorella lets me give the patient a different version of the challenge.

[00:34:57]I put a miniature screwdriver in her blind spot.

[00:34:59]BERLIN: Okay, I'm going to have you open your eyes and fixate.

[00:35:02]Okay.

[00:35:02]Okay.

[00:35:07]I see nothing.

[00:35:08]You see nothing.

[00:35:09]Nothing.

[00:35:09]Okay.

[00:35:10]BERLIN: Even though she says she sees nothing, look at which tool she picks.

[00:35:14]Now turn over there and look at the objects.

[00:35:16]Tell me what you think you saw.

[00:35:17]WOMAN: The screwdriver.

[00:35:17]BERLIN: Yep.

[00:35:20]BERLIN: Now try another one.

[00:35:22]BERLIN: Next, I display a tiny wrench.

[00:35:25]♪ ♪ Did you see anything?

[00:35:31]No.

[00:35:31]No? Okay.

[00:35:33]Look over there and guess what you think you, was there.

[00:35:37]I think the wrench?

[00:35:37]Yep.

[00:35:40]WOMAN: I'm going to guess the scissors.

[00:35:42]Yeah, good job, great, scissors.

[00:35:44]BERLIN: Time and time again, she makes the right choice.

[00:35:47]Amazing, so, you know, it seems to me that you're saying you're not seeing anything, yet when I'm asking you to choose, you're pretty much getting it correct, so something is getting in.

[00:35:58]BERLIN: How is this possible?

[00:36:01]It's as if she sees the tools, but doesn't know it.

[00:36:04]BATTELLI: They actually saw something.

[00:36:05]Mm-hmm, certainly.

[00:36:05]But they're not entirely aware of it.

[00:36:08]Information is getting in, affecting our behavior and how we're responding to the world around us, without there being a conscious perception of that piece of visual information.

[00:36:14]Correct.

[00:36:16]WOMAN: I'm gonna go with the hammer again.

[00:36:18]BERLIN: Until patients like this, we scientists had never seen perception separate from conscious experience.

[00:36:26]And this tells us that perception and consciousness are separate things in the brain.

[00:36:31]But it also has me wondering, if someone can still use visual information without awareness of it, why do we have consciousness at all?

[00:36:39]What is consciousness for?

[00:36:42]A clue might come from babies.

[00:36:44](cooing) ALISON GOPNIK: Babies-- everything we know suggests that they're born conscious.

[00:36:50]They're certainly taking in information from the time they're born.

[00:36:52]REBECCA SAXE: They are making rational choices about what they learn from extremely early on.

[00:36:58]And they are forming memories of their specific surroundings, of their parents, of their important relationships.

[00:37:05]We can see that in their behavior.

[00:37:08]BERLIN: And all that behavior burns a lot of fuel.

[00:37:12]GOPNIK: Brains are expensive computing gadgets.

[00:37:14]So while you're just sitting here, your brain is using up about 20% of all the calories that you have, so it's using up quite a bit.

[00:37:21]But if you think about a two-year-old, his brain is using 60% of his calories.

[00:37:27]So almost all that food is just going to keep his brain going.

[00:37:32]BERLIN: To understand why young brains might need so much more fuel, check out the connections in a toddler's brain versus an adult's.

[00:37:40]A two-year-old's brain has about two quadrillion synapses.

[00:37:44]By the time they hit adulthood, that number is cut in half.

[00:37:49]So if you think about the difference between the baby brain, the child's brain, and the adult brain, the child's brain is more like back country roads where you have little, tiny roads that are going from one village to the next.

[00:38:02]None of them are very efficient.

[00:38:04]There's not a lot of traffic, and the traffic doesn't go very quickly, but they connect lots and lots of different places.

[00:38:09]And the adult brain is more like superhighways that get you from one place to another very quickly, and take a lot of traffic, but don't connect as many different places.

[00:38:19]BERLIN: As we age, in the interest of efficiency, we strengthen the connections that are useful to us and prune the rest.

[00:38:27]MACKNIK: You basically take neurons you don't need, you get rid of them.

[00:38:30]And what you have now is a very lean machine that does certain things and it does them very well.

[00:38:39]GOPNIK: We see this early brain that's very exploratory, that has lots and lots of potential, lots of possibilities.

[00:38:45]Not very good at putting on your jacket and getting out to preschool in the morning.

[00:38:49]And then we have this later brain that's very good at doing things.

[00:38:52]Not so good at changing, not so good at taking in new information, not so good at doing something new.

[00:38:57]BERLIN: What this suggests is that maybe what consciousness is for is choosing what's important for us to be aware of at any given moment.

[00:39:05]Kind of like a spotlight.

[00:39:07]GOPNICK: For adults, it's as if consciousness is this bright spotlight in one place and everything around it is dark.

[00:39:14]BERLIN: While for children and babies, it would be more like a flood light, where nearly everything is illuminated.

[00:39:20]GOPNIK: You're conscious of a lot more that's going on.

[00:39:24]BERLIN: Consciousness may be like an amplifier, boosting the important signals over the noise.

[00:39:29]Is there any evidence?

[00:39:33]This is where FMRI comes in, a special tool in neuroscience that takes pictures of the brain while it's doing something to map where blood flow is in high demand.

[00:39:41]The result is a map of brain activity.

[00:39:45]So where is consciousness in the brain and how does it work?

[00:39:51]To find out, neuroscientists designed a clever series of experiments that go like this.

[00:39:56]They start by flashing a word on a screen for about 30 milliseconds.

[00:40:01](computer beeps) DEHAENE: You flash this word, and the person is not able to see the word at all.

[00:40:06]She says, "There was no word."

[00:40:07]BERLIN: But in the FMRI scanner, the visual cortex is activated, even though people say they don't see anything.

[00:40:15]So the trick here is to find the threshold.

[00:40:18]Find the timing where sometimes people consciously see the image.

[00:40:24]DEHAENE: So if you make now the word a little bit longer, suddenly, the person says, "Oh, well, there is a word, obviously."

[00:40:32]And it's completely visible.

[00:40:34]There is really a sort of all or none phenomenon.

[00:40:36]Either you see it or you don't.

[00:40:38]And once you've done that, you've got a really powerful window onto the neural correlates of conscious perception.

[00:40:45]BERLIN: When that happens, suddenly, a suite of different parts of the brain shows a surge in activity: the parietal cortex, which integrates the senses, the anterior cingulate, which modulates drive and decision-making, and the prefrontal cortex, which handles reasoning and higher-order cognition.

[00:41:03]An ignition of distributed brain areas that come online together, speak to each other, and broadcast this information to the rest of the brain.

[00:41:12]And this is what we think is occurring during conscious perception.

[00:41:17]BERLIN: According to some experts, this communication between brain regions is the signature of consciousness.

[00:41:24]This discovery could have real-world applications in matters of life and death.

[00:41:29]But that would take one more step: figuring out how to measure consciousness.

[00:41:34]♪ ♪ SETH: In science, when we struggle to understand a phenomenon that seems quite mysterious, it's often really important to be able to measure it.

[00:41:44]So a few hundred years ago, this happened with heat-- you know, it was the development of thermometers that catalyzed our understanding.

[00:41:53]Could something work for consciousness the same way?

[00:41:56]Could we have a consciousness-ometer that will lead us to a deeper understanding of what consciousness is?

[00:42:02]BERLIN: That deeper understanding could transform the treatment of people with brain injuries.

[00:42:08]BRIAN EDLOW: So every year, over a million people worldwide will come into an intensive care unit unresponsive, comatose.

[00:42:15]The challenge that we face is that our bedside exam-- asking the person to open their eyes, pinching them and seeing if they'll respond, seeing if they move their arms and their legs-- that bedside exam is fundamentally limited.

[00:42:27]BERLIN: Limited because it often misses people who are actually conscious.

[00:42:32]In this case, we have a healthy volunteer from the "NOVA" team, but what if she were unresponsive?

[00:42:38]How would we ever know if she was conscious?

[00:42:41]Brian Edlow's team at Mass. General Hospital is testing a new technique to find out.

[00:42:46]So tell me, what are you doing here?

[00:42:48]EDLOW: We are pinging the brain with a magnetic pulse and looking for an electrical echo.

[00:42:53]The ping is transcranial magnetic stimulation, or TMS.

[00:42:58]The echo is the key; if it dies out quickly, the patient is unconscious-- they might be in a coma, deep sleep, or under anesthesia.

[00:43:06]If instead the echo rings out across the brain and becomes more complex, the patient is likely to be conscious and aware, even if they appear unresponsive.

[00:43:17]EDLOW: The analogy that we like to use is throwing a pebble in a lake.

[00:43:20]So the pebble represents the TMS pulse to stimulate the brain, and the brain waves, the electrical ripples that emanate from that pulse, represent the waves in the lake.

[00:43:32]The more complex those waves are, and the longer duration, the more likely that person is to be conscious.

[00:43:40]BERLIN: To detect those brain waves, Edlow's team uses E.E.G., a tool that measures electrical activity in the brain, to quantify the amount of complexity a patient's brain bounces back.

[00:43:51]Here's how it works.

[00:43:52]♪ ♪ All neurons, when poked by a magnet, will kick back an electrical signal that looks like this: a brain wave.

[00:44:00]But if the surrounding neurons aren't healthy, those brain waves won't get very far.

[00:44:06]It turns out that in conscious people, even those who appear unresponsive, not only do those brain waves spread all over the brain, but they become more complex too-- it's as if, to use music as the analogy, what starts as a single repeated note by a few neurons eventually turns into a coordinated symphony of millions.

[00:44:28]♪ ♪ DEHAENE: We find is that there is this explosion of complexity only when the person is conscious.

[00:44:36]This complexity, the way different brain areas speak to each other, is a signature, a marker of consciousness.

[00:44:43]BERLIN: Studies of hundreds of patients in various states-- from deep sleep to anesthesia to coma-- have enabled scientists to develop a complexity scale.

[00:44:52]A score above a certain threshold means you are conscious or have the capacity for consciousness.

[00:44:59]EDLOW: Multiple studies have now shown that 15% to 20% of patients who appear unresponsive, they don't express themselves on our behavioral exam, they are actually conscious.

[00:45:09]So, you know, will this be able to help people in those states?

[00:45:13]When we speak to families about what matters to them most, it is that patient's current level of consciousness and their potential for future recovery of consciousness.

[00:45:24]If families were to have that information, it could fundamentally affect the decisions they make about whether to continue life-sustaining therapy.

[00:45:33]DEHAENE: Progress in the clinic is extremely real and fast, and people realize that the problem of consciousness is starting to be solved.

[00:45:41]♪ ♪ BERLIN: Starting to be solved, because while we have several clues about how conscious awareness might work in the brain, this is only the beginning.

[00:45:52]How all of those pieces of brain activation add up to you, a distinct individual with a sense of self, is still a mystery.

[00:46:00]I know my brain creates an internal experience by knitting together bits of sensory information, filling in the gaps with its best guess of what's out there in the world, but what are those guesses based on?

[00:46:12]Memory.

[00:46:14]Each of us has a life rich with experiences to draw from.

[00:46:18]Where we were born, went to school, who we fell in love with.

[00:46:23]Memories are the cornerstone of our identities, but as it turns out, they have a very shaky foundation.

[00:46:30]I could swear by it, and would pass every lie detector test, that... I had met Mother Teresa.

[00:46:36]But I hadn't.

[00:46:37]Something that I wanted to happen but it never did happen.

[00:46:43]SCHILLER: The stories we tell ourselves, or what we consider our memory, is a construction.

[00:46:48]We create these representations.

[00:46:50]And they're very dynamic, they constantly change.

[00:46:53]You're kind of living a revision of the story of your life, constantly.

[00:46:59]SETH: The more often we recall things, the less objectively accurate our memories become.

[00:47:06]BERLIN: It turns out that every time you a recall a memory-- your first kiss, graduating from college, the death of a loved one-- the very act of recollection makes it vulnerable to change.

[00:47:19]SCHILLER: So when you experience a new event, it has to be stored in the brain.

[00:47:23]And then, we used to think that whenever you think about that event, you retrieve the same original memory.

[00:47:30]But what we got to realize in the last few decades is that whenever you retrieve a memory, it goes back to an unstable state.

[00:47:38]BERLIN: In 2000, memory scientist Eric Kandel won the Nobel Prize for showing that each memory creates new synapses, connections that store the memory.

[00:47:49]But what happens when you recall it?

[00:47:53]Every time you remember it, you bring it up into your working memory and you perceive it, and you destroy the long-term memory.

[00:47:59]And you actually have to recast it into long-term memory when you re-remember it.

[00:48:06]So every single time you remember something, you actually add more noise to it, so that it's more and more and more false throughout time.

[00:48:14]BERLIN: This mechanism, called reconsolidation, was first discovered in rodents, where neuroscientists witnessed what happens when a memory gets recollected: for the memory to return to long-term storage, the connections between neurons actually have to get rebuilt.

[00:48:31]Recent experiments have suggested this is likely a mechanism in human brains, as well, because certain drugs known to disrupt reconsolidation have been shown to alter human memories.

[00:48:44]FENTON: We're stuck with the problem of, how do we know what is true?

[00:48:48]How do we know what's real?

[00:48:51]And maybe part of the recognition is, some of those things don't matter as much as we think they do.

[00:49:01]SCHILLER: If we think about the fact that maybe our memories are not as they originally happened, it could be a scary thought, because then, who are we?

[00:49:09]I think you need to think about it as something more liberating, because if you're stuck with original representations, you're kind of stuck in the past.

[00:49:20]BERLIN: Just like our perceptions, our sense of self is dynamic, built to serve us in the present.

[00:49:26]SETH: Our experience of, of self is a construction at all sorts of different levels.

[00:49:31]What the brain is doing, is interested in, is weaving together a kind of story.

[00:49:37]FENTON: The brain is a storytelling machine, right?

[00:49:40]It's a machine that's designed to make predictions.

[00:49:44]KASTHURI: The narratives that we tell ourselves are the biggest illusions that we ever participate in.

[00:49:49]Your sense of who you are is an illusion, as everything else-- you're no exception.

[00:49:56]BERLIN: But if even our sense of self is an illusion, where does that leave us?

[00:50:01]MARTINEZ-CONDE (chuckles): Trust the illusion, that's the only thing that we can be sure of, that what we perceive is not what's there.

[00:50:17]BERLIN: So all these years later, in my quest to understand where my thoughts come from and how my brain works, I've learned that my brain is an exquisite machine that perceives reality in the service of survival, not accuracy.

[00:50:33]The world I carry inside of my head is a construction of my brain built on bits of sensory information woven together with memory to create a conscious experience.

[00:50:45]Now, to some this might sound scary, but to me, it's inspiring.

[00:50:50]♪ ♪ SETH: The simple act of opening our eyes and seeing a world, we should not take that for granted.

[00:50:59]And in realizing what a miracle of neural computation is going on under the hood, to give us even the simplest experiences, I think this adds value, it adds meaning, it adds depth to our lives.

[00:51:14]♪ ♪ DEHAENE: I think it's liberating to understand that we rise from this organization of matter.

[00:51:23]It means that we can be a little bit more humble.

[00:51:25]We are gorgeous machines designed by evolution as well as by our environment, education, friends, families.

[00:51:34]All of that is inscribed in our brains.

[00:51:39]SAXE: Sometimes when I wonder what I'm doing with my life, I think how is it that a spatial and temporal pattern of electrical signals passing between cells in our brains makes us who we are?

[00:51:52]That just being a part of the team asking that question is worth keeping going for.

[00:52:00]♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪ ♪