Visual Storytelling & Future Media: Scott McCloud’s Comic Theory

[00:00:19]Of the five senses, vision is the one that I appreciate the most, and it's the one that I can least take for granted.

[00:00:25]I think this is partially due to my father, who was blind.

[00:00:28]It was a fact that he didn't make much of a fuss about, usually.

[00:00:32]One time in Nova Scotia, when we went to see a total eclipse of the sun -- (Laughter) Yeah, same one as in the Carly Simon song, which may or may not refer to James Taylor, Warren Beatty or Mick Jagger; we're not really sure.

[00:00:43]They handed out these dark plastic viewers that allowed us to look directly at the sun without damaging our eyes.

[00:00:49]But Dad got really scared; he didn't want us doing that.

[00:00:52]He wanted us instead to use these cheap cardboard viewers, so that there was no chance at all that our eyes would be damaged.

[00:00:58]I thought this was a little strange at the time.

[00:01:01]What I didn't know at the time was that my father had actually been born with perfect eyesight.

[00:01:05]When he and his sister Martha were just very little, their mom took them out to see a total eclipse -- or actually, a solar eclipse -- and not long after that, both of them started losing their eyesight.

[00:01:16]Decades later, it turned out that the source of their blindness was most likely some sort of bacterial infection.

[00:01:23]As near as we can tell, it had nothing whatsoever to do with that solar eclipse, but by then my grandmother had already gone to her grave thinking it was her fault.

[00:01:32]So, Dad graduated Harvard in 1946, married my mom, and bought a house in Lexington, Massachusetts, where the first shots were fired against the British in 1775, although we didn't actually hit any of them until Concord.

[00:01:45]He got a job working for Raytheon designing guidance systems, which was part of the Route 128 high-tech axis in those days -- so, the equivalent of Silicon Valley in the '70s.

[00:01:55]Dad wasn't a real militaristic kind of guy; he just felt bad that he wasn't able to fight in World War II on account of his handicap, although they did let him get through the several-hour-long army physical exam before they got to the very last test, which was for vision.

[00:02:11](Laughter) So Dad started racking up all of these patents and gaining a reputation as a blind genius, rocket scientist, inventor.

[00:02:21]But to us he was just Dad, and our home life was pretty normal.

[00:02:25]As a kid, I watched a lot of television and had lots of nerdy hobbies like mineralogy and microbiology and the space program and a little bit of politics.

[00:02:33]I played a lot of chess.

[00:02:35]But at the age of 14, a friend got me interested in comic books, and I decided that was what I wanted to do for a living.

[00:02:41]So, here's my dad: he's a scientist, he's an engineer and he's a military contractor.

[00:02:49]So, he has four kids, right?

[00:02:51]One grows up to become a computer scientist, one grows up to join the Navy, one grows up to become an engineer... And then there's me: the comic book artist.

[00:03:00](Laughter) Which, incidentally, makes me the opposite of Dean Kamen, because I'm a comic book artist, son of an inventor, and he's an inventor, son of a comic book artist.

[00:03:11](Laughter) Right? It's true.

[00:03:14](Applause) The funny thing is, Dad had a lot of faith in me.

[00:03:19]He had faith in my abilities as a cartoonist, even though he had no direct evidence that I was any good whatsoever; everything he saw was just a blur.

[00:03:26]Now, this gives a real meaning to the term "blind faith," which doesn't have the same negative connotation for me that it does for other people.

[00:03:33]Now, faith in things which cannot be seen, which cannot be proved, is not the sort of faith that I've ever really related to all that much.

[00:03:40]I tend to like science, where what we see and can ascertain are the foundation of what we know.

[00:03:46]But there's a middle ground, too -- a middle ground tread by people like poor old Charles Babbage and his steam-driven computers that were never built.

[00:03:56]Nobody really understood what it was that he had in mind except for Ada Lovelace, and he went to his grave trying to pursue that dream.

[00:04:05]Vannevar Bush with his memex -- this idea of all of human knowledge at your fingertips -- he had this vision.

[00:04:12]And I think a lot of people in his day probably thought he was a bit of a kook.

[00:04:16]And, yeah, we can look back in retrospect and say, "Yeah, ha-ha, it's all microfilm -- (Laughter) But that's not the point; he understood the shape of the future.

[00:04:24]So did J.C.R. Licklider and his notions for computer-human interaction.

[00:04:29]Same thing: he understood the shape of the future, even though it was something that would only be implemented by people much later.

[00:04:37]Or Paul Baran, and his vision for packet switching.

[00:04:40]Hardly anybody listened to him in his day.

[00:04:43]Or even the people who actually pulled it off, the people at Bolt, Beranek and Newman in Boston, who just would sketch out these structures of what would eventually become a worldwide network, and sketching things on the back of napkins and on note papers and arguing over dinner at Howard Johnson's -- on Route 128 in Lexington, Massachusetts, just two miles from where I was studying the Queen's Gambit Deferred and listening to Gladys Knight & The Pips singing "Midnight Train to Georgia" -- (Laughter) in my dad's big easy chair, you know?

[00:05:12]So, three types of vision, right?

[00:05:14]Vision based on what one cannot see, the vision of that unseen and unknowable.

[00:05:20]The vision of that which has already been proven or can be ascertained.

[00:05:23]And this third kind, a vision of something which can be, which may be, based on knowledge but is, as yet, unproven.

[00:05:33]Now, we've seen a lot of examples of people who are pursuing that sort of vision in science, but I think it's also true in the arts, it's true in politics, it's even true in personal endeavors.

[00:05:43]What it comes down to, really, is four basic principles: learn from everyone; follow no one; watch for patterns; and work like hell.

[00:05:52]I think these are the four principles that go into this.

[00:05:55]And it's that third one, especially, where visions of the future begin to manifest themselves.

[00:06:00]What's interesting is that this particular way of looking at the world, is, I think, only one of four different ways that manifest themselves in different fields of endeavor.

[00:06:09]In comics, I know that it results in sort of a formalist attitude towards trying to understand how it works.

[00:06:15]Then there's another, more classical attitude which embraces beauty and craft; another one which believes in the pure transparency of content; and then another, which emphasizes the authenticity of human experience and honesty and rawness.

[00:06:29]These are four very different ways of looking at the world.

[00:06:32]I even gave them names: the classicist, the animist, the formalist and iconoclast.

[00:06:36]Interestingly, they seem to correspond more or less to Jung's four subdivisions of human thought.

[00:06:41]And they reflect a dichotomy of art and delight on left and the right; tradition and revolution on the top and the bottom.

[00:06:48]And if you go on the diagonal, you get content and form, and then beauty and truth.

[00:06:52]And it probably applies just as much to music and movies and fine art, which has nothing whatsoever to do with vision at all, or, for that matter, nothing to do with our conference theme of "Inspired by Nature," except to the extent of the fable of the frog who gives a ride to the scorpion on his back to get across the river because the scorpion promises not to sting him, but the scorpion stings him anyway and they both die, but not before the frog asks him why, and the scorpion says, "Because it's my nature."

[00:07:18]In that sense, yes.

[00:07:19](Laughter) So this was my nature.

[00:07:24]The thing was, I saw that the route I took to discovering this focus in my work and who I was -- I saw it as just this road to discovery.

[00:07:35]Actually, it was just me embracing my nature, which means that I didn't actually fall that far from the tree, after all.

[00:07:43]So what does a "scientific mind" do in the arts?

[00:07:48]I started making comics, but I also started trying to understand them, almost immediately.

[00:07:53]One of the most important things about comics that I discovered was that comics are a visual medium, but they try to embrace all of the senses within it.

[00:08:01]So, the different elements of comics, like pictures and words, and the different symbols and everything in between that comics presents, are all funneled through the single conduit, a vision.

[00:08:11]So we have things like resemblance, where something which resembles the physical world can be abstracted in a couple of different directions: abstracted from resemblance, but still retaining the complete meaning, or abstracted away from both resemblance and meaning towards the picture plane.

[00:08:26]Put all these three together, and you have a nice little map of the entire boundary of visual iconography, which comics can embrace.

[00:08:33]And if you move to the right you also get language, because that's abstracting even further from resemblance, but still maintaining meaning.

[00:08:41]Vision is called upon to represent sound and to understand the common properties of those two and their common heritage as well; also, to try to represent the texture of sound to capture its essential character through visuals.

[00:08:57]There's also a balance between the visible and the invisible in comics.

[00:09:02]Comics is a kind of call and response, in which the artist gives you something to see within the panels, and then gives you something to imagine between the panels.

[00:09:11]Also, another sense which comics' vision represents, and that's time.

[00:09:17]Sequence is a very important aspect of comics.

[00:09:20]Comics presents a kind of temporal map.

[00:09:24]And this temporal map was something that energizes modern comics, but I was wondering if perhaps it also energizes other sorts of forms, and I found some in history.

[00:09:34]You can see this same principle operating in these ancient versions of the same idea.

[00:09:40]What's happening is, an art form is colliding with a given technology, whether it's paint on stone, like the Tomb of Menna the Scribe in ancient Egypt, or a bas-relief sculpture rising up a stone column, or a 200-foot-long embroidery, or painted deerskin and tree bark running across 88 accordion-folded pages.

[00:09:57]What's interesting is, once you hit "print" -- and this is from 1450, by the way -- all of the artifacts of modern comics start to present themselves: rectilinear panel arrangements, simple line drawings without tone, and a left-to-right reading sequence.

[00:10:11]And within 100 years, you already start to see word balloons and captions, and it's really just a hop, skip and a jump from here to here.

[00:10:19]So I wrote a book about this in '93, but as I was finishing the book, I had to do a little bit of typesetting, and I was tired of going to my local copy shop to do it, so I bought a computer.

[00:10:29]And it was just a little thing -- it wasn't good for much except text entry -- but my father had told me about Moore's law back in the '70s, and I knew what was coming.

[00:10:38]And so, I kept my eyes peeled to see if the sort of changes that happened when we went from pre-print comics to print comics would happen when we went beyond, to post-print comics.

[00:10:49]So, one of the first things proposed was that we could mix the visuals of comics with the sound, motion and interactivity of the CD-ROMs being made in those days.

[00:10:57]This was even before the Web.

[00:10:59]And one of the first things they did was, they tried to take the comics page as is and transplant it to monitors, which was a classic McLuhanesque mistake of appropriating the shape of the previous technology as the content of the new technology.

[00:11:11]And so, what they would do is have these comic pages that resemble print comics pages, and they would introduce all this sound and motion.

[00:11:18]The problem was that if you go with this basic idea that space equals time in comics, what happens is that when you introduce sound and motion, which are temporal phenomena that can only be represented through time, they break with that continuity of presentation.

[00:11:35]Interactivity was another thing.

[00:11:36]There were hypertext comics, but the thing about hypertext is that everything in hypertext is either here, not here, or connected to here; it's profoundly nonspatial.

[00:11:44]The distance from Abraham Lincoln to a Lincoln penny to Penny Marshall to the Marshall Plan to "Plan 9" to nine lives: it's all the same.

[00:11:51](Laughter) But in comics, every aspect of the work, every element of the work, has a spatial relationship to every other element at all times.

[00:12:01]So the question was: Was there any way to preserve that spatial relationship while still taking advantage of all of the things that digital had to offer us?

[00:12:09]And I found my personal answer for this in those ancient comics that I was showing you.

[00:12:14]Each of them has a single unbroken reading line, whether it's going zigzag across the walls or spiraling up a column or just straight left to right, or even going in a backwards zigzag across those 88 accordion-folded pages, the same thing is happening; that is, that the basic idea that as you move through space you move through time, is being carried out without any compromise, but there were compromises when print hit.

[00:12:36]Adjacent spaces were no longer adjacent moments, so the basic idea of comics was being broken again and again and again and again.

[00:12:43]And I thought, OK, well, if that's true, is there any way, when we go beyond today's print, to somehow bring that back?

[00:12:52]Now, the monitor is just as limited as the page, technically, right?

[00:12:57]It's a different shape, but other than that, it's the same basic limitation.

[00:13:01]But that's only if you look at the monitor as a page, but not if you look at the monitor as a window.

[00:13:08]And that's what I propose, that perhaps we could create these comics on an infinite canvas, along the X axis and the Y axis and staircases.

[00:13:17]We could do circular narratives that were literally circular.

[00:13:20]We could do a turn in a story that was literally a turn.

[00:13:23]Parallel narratives could be literally parallel.

[00:13:27]X, Y and also Z.

[00:13:30]So I had all these notions.

[00:13:31]This was back in the late '90s, and other people in my business thought I was pretty crazy, but a lot of people then went on and actually did it.

[00:13:38]I'm going to show you a couple now.

[00:13:41]This was an early collage comic by a fellow named Jasen Lex.

[00:13:47]And notice what's going on here.

[00:13:49]What I'm searching for is a durable mutation -- that's what all of us are searching for.

[00:13:54]As media head into this new era, we are looking for mutations that are durable, that have some sort of staying power.

[00:14:03]Now, we're taking this basic idea of presenting comics in a visual medium, and we're carrying it through all the way from beginning to end.

[00:14:11]That's that entire comic you just saw, up on the screen right now.

[00:14:14]But even though we're only experiencing it one piece at a time, that's just where the technology is right now.

[00:14:19]As the technology evolves, as you get full immersive displays and whatnot, this sort of thing will only grow; it will adapt.

[00:14:27]It will adapt to its environment; it's a durable mutation.

[00:14:33]Here's another one.

[00:14:34]This is by Drew Weing; this is called "'Pup' Ponders the Heat Death of the Universe."

[00:15:04]See what's going on here as we draw these stories on an infinite canvas is you're creating a more pure expression of what this medium is all about.

[00:15:21]We'll go by this a little quickly. You get the idea.

[00:15:24]I just want to get to the last panel.

[00:15:32][Cat 1: Pup! Earth to Pup! Cat 2: Come play baseball with us!]

[00:15:35](Laughter) [Pup: Did either of you realize that eventually the universe will be nothing but a thin, cold gas spread across infinite, lonely space?]

[00:15:43][Cat 1: Oh... Cat 2: We'd better hurry, then!]

[00:15:46](Laughter) Just one more.

[00:15:53]Talk about your infinite canvas.

[00:15:55]It's by a guy named Daniel Merlin Goodbrey, in Britain.

[00:15:58]Why is this important?

[00:16:01]I think this is important because media -- all media -- provide us a window back into our world.

[00:16:08]Now, it could be that motion pictures and eventually, virtual reality, or something equivalent to it, some sort of immersive display, is going to provide us with our most efficient escape from the world that we're in.

[00:16:22]That's why most people turn to storytelling, to escape.

[00:16:24]But media provides us with a window back into the world we live in.

[00:16:31]And when media evolve so that the identity of the media becomes increasingly unique -- because what you're looking at is comics cubed, you're looking at comics that are more comics-like than they've ever been before -- when that happens, you provide people with multiple ways of reentering the world through different windows.

[00:16:53]And when you do that, it allows them to triangulate the world they live in and see its shape.

[00:16:58]That's why I think this is important.

[00:17:00]One of many reasons, but I've got to go now.

[00:17:02]Thank you for having me.