Ray Tracing Glass Surfaces and Caustics: A Coding Adventure

[00:00:00][Music] Hello everyone and welcome to another episode of coding adventures. Today I want to return to the ray tracing project to add in some glassy features and also focus a bit on rendering some nice costics.

[00:00:14]So opening the project we have the simple scene set up and I'll run the render quickly to see how that looks.

[00:00:21]Okay, clearly there is a disturbing lack of glass anyway in this room. So we'd better get to work. Although to be totally transparent, a lot of what we need we'll simply be snagging from an earlier episode on fluid rendering since if you happen to have watched it, you might recall this ray matching approach we tried where we took Snell's law into account which describes how light changes direction as it plunges into the water and also when emerging on the other side. We looked a little into the maths then of how this depends on the speed that light can travel in different media. Like in water for instance, it's forced to slow down by about 25% compared to air. And glass is even harder to swim through if you can believe it. Slowing light down by a whopping 33% there, more or less for different types of glass. But anyway, we have in the code here this calculate reflection and refraction function which lets us specify the direction of the incoming light as well as the direction that the surface it's about to smash into is facing. and finally two refractive indices describing the speed of light on either side of that surface as ratios to the speed of light in a vacuum. This then gives us back a little light response structure holding the two directions the light can go in either reflecting off the surface or refracting through it or both. We can have some fraction going one way and the rest going the other. That bifocation of the light is a bit bothersome, especially for real-time rendering. Like we had to make some serious concessions for this interactive fluid, such as only following the path that most of the light was taking.

[00:01:52]Thankfully, however, our rate tracer is already far from real time, unless you're happy to fill in a lot of blanks with your imagination. So, I don't really have any qualms about making it even slower. So let's head into the code for that then where we have this ray structure for storing things like the position and direction of each ray that we fire out from the camera. And to handle these rays now branching out into different directions. I typically reach for a recursive approach. But our shader language here sadly doesn't support that. We did actually run into this exact same problem the last time we were working on this project though when we wanted to recursively search through a hierarchy of bounding boxes as an efficient way of detecting collisions.

[00:02:30]So, we can just recycle that solution, which was to simply store all the stuff we still want to explore in a stack.

[00:02:36]Last time that was bounding boxes, and this time its rays, beginning, of course, with the initial array coming from the camera. We then just have a loop that keeps on popping rays off the top of the stack until it's empty and runs for each of those this main path tracing loop, which basically bounces the ray around the world to figure out how much light can reach the camera along that path. So, we're going to want to change this behavior if our array runs into an object made of glass, which we can test for by looking at its material properties since this has a flag for defining different types of materials, which is currently only used for specifying which walls should have the simple checkerboard pattern applied.

[00:03:14]But we can just decree that a value of two now indicates a glass-like material.

[00:03:20]If that's what we've hit, we'll want to know if that means the ray is hitting it from the outside or from within, which the orientation of the triangle could tell us. And so we can then get the appropriate index of refraction on either side of that surface. For now, at least simply assuming that outside the object is empty. And from that, we can calculate our directions of reflection and refraction. Figure out which one the majority of light is going to take, and redirect our current ray to go along there. This time though, we'll also consider how much light is going along the less popular path. And as long as there's at least a little and we have space on our stacks tool to store it, we'll create a new array pointing that way and push it onto the stack to look at later. With that, we should be done.

[00:04:01]Except I almost forgot that this backface value doesn't actually exist yet. So, I'll add it to the structure.

[00:04:06]And then over where we're detecting triangle intersections, we can use the dot product to see if the direction the ray is traveling and the direction that the triangle is facing are less than 90° apart, which would mean that the ray is colliding from the rear end. We should also add a parameter to say whether we actually want these back faces to be detected. And if so, we just need to use the absolute value over here so that the intersection is always considered valid unless the ray is extremely close to parallel with the triangle, in which case we can't really trust the result due to limited floatingoint precision.

[00:04:39]Running this at first, I got some interesting glitchy patterns where it's now actually slamming into the back face of our invisible wall, which I'd kind of forgotten about. It is mysterious though how the pattern is slightly different from left to right and not there at all in the upper half. You've really got to wonder sometimes what the GPU is getting up to in there.

[00:05:00]I see how it is. Well, I think what I'll do is tell it to only consider backface collisions for glassy objects so that the rays just glide right through our one-sided wall again.

[00:05:11]Then let's mark these three objects here as glass. Give them a refractive index somewhere around 1.5 and see how it goes.

[00:05:22]Oh, my computer died.

[00:05:25]Well, after a brief bit of necromancy and plopping this comparison inside the parenthesis where it was supposed to be.

[00:05:30]I'd rather not dwell on how long it took me to notice that. We can now try running this again. And just like that, we have successfully created obsidian.

[00:05:39]That's a kind of glass, right? Okay, perfect. I guess we're done then. Well, all right. One thing I have realized at least is that the normal vector we're calculating here should actually be flipped around if it's a back face that the ray is hitting so that it's pointing away from the direction of impact like our reflection and refraction functions expect. And with that fixed, we can at last see through the glass. Well, sort of. It's it's progress at least.

[00:06:06]We are still seeing these ring patterns we had on the back wall before, though.

[00:06:10]And I think the solution to these is to simply offset the ray's position of impact by a tiny amount along the normal, either forwards or backwards, depending on whether it's reflecting off or refracting through the surface, just so that it's unambiguously on one side or the other to avoid any numerical confusion. All right, let's run the render again with that fix in place. And this is now looking vastly more glassy if you ask me. But it still doesn't feel quite right. Though to be fair, glass is kind of confusing, and I'm not entirely confident I'll know what is working when I see it. So, I think it's high time we got a little crafty here.

[00:06:47][Music] [Music] Isn't it beautiful? Also, our reference is showing very clearly how the red wall on the left should actually be visible on the right side of the glass and vice versa for the green wall, which was the wrong way around in our render. And fortunately, that problem didn't take too long to find. I just got the back face condition back to front in the code. So the computer thought the air was made of glass and the glass was made of air. Having cleared up that minor confusion, we should now be seeing some red on the right and green on the left, which we do indeed.

[00:07:43]But taking another look at the reference, there is another discrepancy here since we can also see the walls very clearly along the base of the cube, which is not the case in our render.

[00:07:55]I think we can maybe fix this though simply by levitating the objects ever so slightly. Essentially just making a little air gap to ensure that the light does actually hit the base of the glass first and not slam directly into the floor. All right, let's see if that works. And it's looking promising, I'd say. We can certainly see the blue wall at the back reflected in the bottom of the boxes. And we also see a bit of light spilling out around the base here, which was visible in our reference as well. Let's actually try lifting these further up into the air so we can get a better sense of how the light is shining through.

[00:08:31]Okay, I believe that this is working as we want now, but maybe we can test a bit better by actually lining the scene up with the reference. So, just a single glass cube at around 45°. And I'll even bevel the edges a bit to match reality.

[00:08:45]And now we can take a look at the reference and compare that directly to the result from our rate tracer. I think it's looking pretty good. Let's see it spinning around. And I am noticing some flickering along those thin edges. So, we'll need to investigate that at some point. But for now, I'm quite happy with the result.

[00:09:04]I would like to try testing out some different refraction indices, though.

[00:09:07]maybe on this giant version of the dragon here. So, starting at one, the rays refract right through it without any deviation, rendering it entirely invisible. Well, aside from these suspicious speckles suddenly popping up all over the place. My best guess is that these are the nefarious nans my nan warned me about. Basically, non-n numbers that crop up when you do naughty things with maths. We can verify that quickly by just checking if any of the light values we are returning are nan and instead returning a super bright green in that case. That way your nan popping up for even a single frame will show up clearly in our accumulated render. So we should see those appearing any moment now.

[00:09:47]Any any moment now.

[00:09:50]Uh never mind. I guess it's something else. Actually hang on. I remember reading once about how the compiler likes to pretend that nans don't exist.

[00:09:58]so the GPU doesn't have to spend time checking for them and handling them properly. And as a result, the nan test we've been using gets optimized away as always false. So I've just looked up quickly how they're actually represented. Basically, these eight exponent bits must all be set to one, and the remaining 23 on the right can just be anything other than zero since zero is used in that case to indicate infinity. We can then simply examine those bits ourselves to screen for any nans trying to sneak past. And now there's no shortage of green speckles to see.

[00:10:31]Having confirmed that as the cause, it was reasonably short work to track them back to this reflectance function, which is what calculates the proportion of light being reflected versus refracted.

[00:10:40]And they're arising from a division by zero over here in the rather strange case that the ray is moving away from a surface, meaning it shouldn't actually be interacting with it at all. I think this is maybe caused by precision problems with the interpolated normals around extremely sharp edges. And since this dragon comes from a 3D scan, there is a decent amount of dodgy geometry.

[00:11:03]But I'm not entirely sure. At any rate, we can just add a little check in here to avoid that division by zero. And with that, our dragon has been rendered invisible. So let's try gradually increasing the index of refraction. And all of a sudden, there it is. The refractive index is all the way up at three now, by the way, which is fairly extreme. Something like diamond, for instance, is only around 2.4. But anyway, I think it's cool to just see how it sort of melts away into thin air as that value goes back down to one.

[00:11:35]I'd also like to see how our glassy dragon looks spinning around, but it's a bit of a tight fit in here at the moment. So, let's just expand the room a little. And now we should be good to go.

[00:11:45]So, after a brisk few hours of rendering, here's what we've got.

[00:11:51]This is looking quite nice, I'd say, but it would be good to have a bit more control over the appearance of the glass. Like, all we can change at the moment is how strongly it refracts the light. But all the color comes entirely from the world around it. So, I'd like to be able to give it some color of its own. To do that, let's jump over to where we're detecting our glass-like materials and add in a line to see if we've just hit a back face, meaning that the ray has been traveling some distance through the object and is about to exit on the other side. In this case, we'll want some of the light to have been absorbed along the way by whatever colorant is in the glass. And so, we can say that the transmittance of this ray is reduced by some factor depending on the travel distance and the particular absorption coefficients of the colorant.

[00:12:36]This function here, by the way, just gives us an exponential fall-off in strength that looks like this as the distance increases. And it's something we've used again and again when rendering clouds and atmospheres and so on. The idea is just that as the light moves through some medium, a tiny fraction can be absorbed or scattered away for every little bit of distance it travels. This means though that the amount being lost is constantly decreasing as well simply because there's less light left for the absorption or out scattering to happen to. So the exponential decay just accounts for this concept.

[00:13:10]All right. So we could say for instance that red light should be absorbed just a little bit. Green light maybe can be absorbed quite a lot and blue light let's say least of all. And if we run this now we can see that our dragon has turned up pretty sort of purple. I suppose we could also multiply those coefficients by some constant to make the light be absorbed more rapidly. So now only the thinnest parts of the dragon are still perfectly see-through, such as these spiny bits over here.

[00:13:36]Let's see how it looks in some other colors as well, just for fun. So here's red and green and blue. And of course, if we just have all three wavelengths be absorbed in equal amounts, then we get this beautiful black dragon. Beyond changing the color though, I think it'd be cool if we could also make the glass more cloudy looking. Kind of like a frosted glass, I guess. So, we need the surface to be roughened up. We already know how to go from a smooth surface to a rough one. We just bounce the rays off in random directions if it's totally rough and blend towards a perfect reflection vector the smoother we want it to seem. So, let's just do the exact same thing in the glass code over here.

[00:14:14]Applying it now to our new refraction ray as well, of course.

[00:14:19]Okay, then starting out fully smooth like before, let's now gradually decrease that smoothness value. And we can see how the light has become a lot more diffuse. We're at around 65% smoothness here, by the way. So, we can still sort of see the back wall through the dragon. It's just very blurry, so the checkerboard pattern is hard to make out. But let's try our turning smoothness all the way down to zero to see what that looks like. And clearly it's no longer at all transparent, but it does still have a nice sense of translucency to it. For contrast, if we disable the glass flag, this is what it looks like instead. And here's the glass. Again, I don't have a real frosted glass dragon to compare this to, a fact which fills me with great sadness. But looking for a reference online, I came across this handsome pair of horses hanging out in Morirano. And I'd say the frosted effect we see here lines up reasonably well with our results.

[00:15:12]Let's return to 65% smoothness, though, since I think it's nice to have a hint of background showing through. And then let's also add a bit of absorption back in to create some color again.

[00:15:22]Something I'm curious to try quickly is to actually separate the smoothness settings for reflections and refractions. So, we could keep the blurry refractions, for instance, but have a sort of smooth reflective coating over top to give us these crisp highlights. I'm not entirely sure what manner of material we're mimicking here, but I think it looks kind of nice nonetheless.

[00:15:44]Anyway, I might not have a real dragon made of glass, but I could at least 3D print a little plastic one. So, I've been haphazardly plopping down some support structures. I'm not terribly experienced for all this, but we can see here the printing plan, and hopefully it will pan out okay.

[00:16:03]All right, that's looking pretty promising. So, let's take a closer look.

[00:16:07][Music] And it is a little rough around the edges, especially by the jaw. I think it's missing most of its teeth, but it's rather majestic anyway.

[00:16:18]So, let's pop it into its house. And just for fun, I've also quickly made a little render to compare.

[00:16:24][Music] I think that lines up really nicely.

[00:16:28]Then, getting a bit carried away, I've also printed out a partner in pink. And let's do a quick switcheroo to see how that one looks. Although actually I'm less interested in looking directly at it and more interested in viewing it through this glass globe I've got over here. This is maybe a bit close up though. We can't really see much. So let me slide it back a bit.

[00:17:05]All right. So, we can see the entirety of our pink dragon reflected in there, or refracted, rather, I guess, since we are seeing through to the other side.

[00:17:13]It's just all flipped around. And I'd like to take a quick look at what's actually happening to the light over here.

[00:17:19]So, I've made a very crude little two-dimensional ray tracer of sorts.

[00:17:23]basically just firing out a bunch of rays to test if they run into this glass sphere here and drawing them as simple line segments so we can see what's going on. If they do hit the sphere, we just calculate the reflections and refractions and recursively follow those rays until either some depth limit is reached or there's just very little light left.

[00:17:43]All right, let's try it out. So, we have our rays shooting out from the camera and refracting through the sphere, causing them to bend inwards and then refracting again through the other side as well, bending them even more. And we can see how they're actually crossing over one another now, just a short distance outside the sphere. So, if we imagine our model over here, we can see why it would appear upside down since the rays at its feet are from the top half of the camera and vice versa. And looking at these rays, I suppose if we move it closer to the sphere, we should see it magnified essentially down to a single point where these light rays kind of converge before suddenly flipping around to its regular orientation when it reaches this little region where the rays haven't crossed over yet.

[00:18:24]Let's try this out in the real world quickly, although the dragons may be a bit big. So instead, I'll write a super secret message here. And we can see how that starts out small and flipped around, but as I slide it closer to the sphere, it gets very big very fast and then suddenly wraps around revealing the message.

[00:18:42]All right, I thought that was interesting, but let's go back to this little scene we set up. And I want to try now rendering this in our ray tracer just to be sure that it works as expected. So, we'll need a ball. And we had perfect bowls in the first episode using a special ray sphere intersection function, but since we have everything set up for triangles now, I guess I'll just generate a highresolution sphere mesh in Blender and plop that into our scene instead. It would be nice at some point to reintegrate support for geometric primitives, though. But anyway, here's what that's looking like with a glass material applied. And it seems fine at first glance, but then I noticed this oddly dark rim around the right side and a weird little notch in the tiles in the middle.

[00:19:22]I went on a few wild goose chases trying to figure this out and ended up making this little test shader here that calculates the correct normal vector for a sphere just by looking at the local position of the current fragment and then comparing that to the normal stored in the mesh. And if we magnify that difference a few thousand times, we can suddenly see some abnormalities in the normals. It looks like at the 12 pentagons one finds in a nikosphere, we get a low error along the lines connecting them together, but a fairly high error everywhere else. So, I tried just switching to my own sphere generator that I made a while back for some procedural planets, but that had issues of its own. Then, after a few more highly unhelpful visualizations to try and understand the issue, I finally realized that I'd neglected to enable smooth shading on the mesh, which essentially averages the normals around each vertex. And while I still don't quite understand what was going on before, that did finally drop the error down to zero. Well, close enough. At any rate, if we crank the multiplier all the way up to a million, there's unsurprisingly still a tiny amount of error there. I don't know why it looks like this exactly, but it's always fun when the numbers make pretty patterns.

[00:20:31]Unless what we've uncovered is more sinister, like the insignia of some secret CGI society.

[00:20:44]Either way, the discrepancy should be far too tiny to cause a visual disturbance anymore, taking us from this wonky result to this, which is now looking nice and symmetrical.

[00:20:55]So, let's at last render the pink dragon and then roll in the ball.

[00:21:03]Focusing in on that, we can see the scene is flipped around in there indeed.

[00:21:06]And while I haven't lined it up terribly well with reality, just using it as a rough reference, I'd say it seems to be working pretty nicely. However, I just discovered that I've made things way more complicated than they need to be because while waiting for that rolling ball animation to render, I took a quick look at this rendering book I've seen recommended and in particular the chapter on dialectrics, which is a category of materials like glass and water for which we need to handle refraction. There's a lot of interesting stuff in here to explore in the future, but what caught my eye was this little line about choosing with probability based on the Fresnel equations. So, what I realized in that moment, as many of you may have been thinking, is that all of this stack stuff we've been doing to mimic recursion is totally unnecessary because the far simpler solution is that say for example 90% of the light is refracted and 10% reflected. Rather than creating a new ray so we can explore both directions, we'll just say that there's a 10% chance of the ray reflecting, otherwise it will be refracted instead. Honestly, it's really funny and a bit distressing that I didn't think of this originally, considering that on essentially the next line of code, we already use the exact same idea for exploring the specular and diffuse directions in the first episode.

[00:22:19]But anyway, it's a good reminder to myself to not just run with the first thing that pops into my head, but to take a moment to think of alternatives before beginning. Just to be fully certain that this does converge to the same result though, I've run it on this test scene here and then compared that to a render with our old implementation.

[00:22:37]They are very similar. But zooming in, I did notice that the new version, aside from being a bit less noisy since it runs almost twice as fast, also has some faint reflective details that the old version was missing since we were ignoring paths with very low amounts of light. So, this is just better in every way, I think.

[00:22:55]Anyway, I've been playing around a bit more with glass balls because I really like the way that they form these focus spots of light on the floor. Like, check this out for instance. These are called costics, which is just a fancy term for when light is reflected or refracted by a curved surface, causing the rays to cross over and create these concentrated clusters of light. And we can see how these spots become more or less blurry depending on the distance.

[00:23:20]Let's actually hop over to that two-dimensional thing we made so we can see a bit better what our light rays are getting up to. And instead of picturing it from the camera's perspective, let's imagine them radiating out from something like a light bulb. This time I'll raise the ray count to around 20,000 so it looks more like a continuous field. And let's have a look at how that interacts with the sphere.

[00:23:43]All right. So we can imagine if the light is hitting the floor or a wall I guess somewhere over here then it would make a bright ring of light with the inside being somewhat dimmer. And then if the wall were further away we can see how that would become focused into a single searing point before finally becoming blurry and blurriier. At some point, I'd like to try adding this sort of light path visualization into our actual ray tracer. But for today, I've just been thinking of some easier ways we could get a similar effect. For instance, here's a scene with a single sphere, and we can see that bright ring of light it's casting on the floor.

[00:24:15]Although, this does have a bright bubble in the middle, which we didn't see in 2D, but I believe that's just because the light source here isn't a perfect point, but more of a tiny bulb since I didn't want it to take forever to render. But anyway, the idea now is to gradually change the distance of the floor over time, essentially sampling many slices of the volume of light that's being refracted through our glass globe. First though, I want to add a long exposure setting to the code here, so that when a frame is finished, instead of resetting the image to a blank slate for the next frame, we can choose to keep the shutter open, so to speak. And so now if we let the floor drop down, the long exposure shot gives us a nice sense for how the light is traveling through the air here.

[00:24:57]Maybe a better approach to try though is simply adding a bit of mist into the scene. So for that, we just need to define some density value from which we can probabilistically determine how far the current light ray would travel before it runs into one of the tiny water droplets that make up the mist. If that's closer than the object the ray would otherwise have collided with, then we can just update the ray's position to this imaginary droplet and send it scattering off in a random direction.

[00:25:24]Okay, let's now render a scene with a glass sphere again and then just increase that density setting to fog things up in here. That's looking good, I think. So, let's spice it up with some more randomly placed spheres of different sizes. and rendering that again. I really like how the mist reveals these regions of space that are experiencing heavy light traffic.

[00:25:43]Now, at the moment, we're mainly looking at costics caused by refraction, but we can just as easily have reflective costics, of course. For instance, if we go back to our 2D view and move the light source along to be inside the circle, suddenly we get this cool cardioid costic cropping up, which is something you've no doubt noticed in many mugs and bowls and anything vaguely cylindrical really.

[00:26:07]Let's slow down the light in our visualization so we can see how the shape develops over time.

[00:26:13]It's maybe actually easier to see what's going on with fewer rays though, so we can see the individual reflections.

[00:26:19]What we're getting here looks a lot more detailed than one would actually see in a coffee cup for instance. But if we roughen up the surface slightly and increase the amount of light lost with each reflection, this is now looking a lot closer to reality.

[00:26:33]I think it would be fun to see this effect in our ray tracer as well. So, I've spent a few minutes modeling a little mug. And let's bring that into our rendering room. Here's how it's looking at the moment with a little light bulb just hovering nearby. So, let's then make the material slightly reflective, which should cause the cardioid to appear. And indeed, there it is.

[00:26:52]Just for fun, I've also written a function to animate the light source soaring in an arc above the mic. And let's render that out.

[00:27:05]Okay, I think this costic looks very cool, but it is hard to beat the costic swan sees in the sea and other watery places since there the curvature of the surface is constantly changing, giving us these lovely shifting patterns of light. If we make a crude mockup in two dimensions, we can get a rough idea of how the waves might focus the rays. And let me add in some more again to smooth this out. Then we can watch how these move as the waves wobble along.

[00:27:32]In the real-time water rendering from a while back, we unfortunately don't see any of these costics, but it is still something I want to try to do there.

[00:27:39]However, in our ray tracer today, it should just work out of the box since we haven't made so many simplifications here. To try it out, let's set up the simulation. Maybe rotating the spawn cube slightly just to give us some more interesting waves perhaps. And let's see how that goes.

[00:27:55]What we have here is the density field of the fluid that's been turned into a mesh with matching cubes. And I think it'll be easiest to write all these meshes out to a bunch of files that can then be loaded into the ray tracer rather than actually merging the two projects together. So I've written this function here to do that. basically just reading the vertex data back from the GPU buffer where it's stored and then writing that to a binary stream in a half precision format just to save on some hard drive space because I'm running low and many of these meshes have millions of vertices.

[00:28:26]That data is then also run through a compression algorithm and finally written to disk. The first time running this, it sprung a leak and rapidly ran out of RAM. But after patching things up, we've now got about a,000 frames of fluid stored in these files here, which fills up around 5 GB in total. Still, after importing one of them into the ray tracing project and having a careful look around, I couldn't shake the feeling that something had gone subtly wrong in the process. Eventually, it dawned on me though that I'd just forgotten to actually decompress the data when loading the file. So, with that fixed, we can correctly display any of the frames. And let's try rendering one now.

[00:29:06]That is looking pretty watery, I'd say.

[00:29:07]But unfortunately, we can't see any light shining through onto the floor because the bottom of the water is acting like a mirror. And so, we just see this reflection instead. Early on, we noted that this was the correct behavior for the glass cube. But I want to test if it is actually how the water should be behaving as well. So, I've got a little vessel over here, and since that's made of glass, we can see our two dragons reflecting off the bottom as expected. But now, let's pour in some water.

[00:29:34][Music] All right, not super surprisingly, I guess. The reflection remains, although it has changed a bit of course due to the different refraction indices at play. But then again, in our scene here, I'm not picturing the water as being inside a glass container, but rather touching the walls and floor of this room directly. And those are made of, I don't know, maybe concrete or some kind of painted plaster, I suppose. So, let's try replacing the floor of our tank with something vaguely similar. And we can see that the reflections do now disappear, allowing us to see the light that's shining down on the bottom.

[00:30:08]Honestly, I'm actually a little confused by this though because why do we get no reflections when the water is touching the floor and yet do when the glass is the one touching the floor? I suppose it must be a matter of how intimately they're connected. Like maybe we could imagine a thin layer of air between the glass and floor that's causing the light to reflect. I'm curious what happens then if we grab some oil for instance, which has a similar refractive index to glass, and use that to try and form a more flush connection to the floor.

[00:30:40]Okay, still seeing a reflection here. Am I just not pushing hard enough?

[00:30:45]I don't know. But from a more forgiving angle, interestingly, we do actually see a big difference. Like at the moment, we have the back wall reflected off the bottom face of the glass. But as it comes into this goopy contact with the red floor, we can actually see that red showing through instead.

[00:31:01]All right, that's enough of my sloppy science experiment. Not a total success, but we have at least seen that we should do the opposite maneuver for the water as what we did for the glass. Moving the model down a little so that it clips through the floor, ensuring that the light is able to bounce diffusely rather than reflecting there. Okay, let's try rendering this now. And we can indeed see the floor this time. And excitingly, the cost sticks we've been looking for are visible there as well. So, after leaving this overnight to render our thousand frames, here's how it looks in motion.

[00:31:36]I think the costics, while somewhat subtle and not entirely in focus, do still add quite a lot to the overall effect.

[00:31:43]Let's try another render. And for this one, I added support for spherical spawners to the simulation just to spice things up. I think it would be fun to give the water a sort of frosted glass appearance this time like we were experimenting with earlier. So, let's see how that goes.

[00:32:01]I didn't intend for this light bulb gizmo to pop up, by the way. That was an accident. But otherwise, I think it looks pretty cool. Maybe we can do one last one with a bit of absorption applied as well.

[00:32:13][Music] All right. There's obviously plenty of room for improvement, but I'm happy that we've been able to capture some watery costics here. Of course, there is another more complex kind of costic that water can create, which is the rainbow.

[00:32:30]So, let's think about just a single drop of water for a moment, and we might as well return to two dimensions for this.

[00:32:36]Now, for a rainbow, our light source will likely be the sun. And while it does radiate out in all directions, the Earth is so small and far away, of course, that the rays hitting us are all pretty much parallel. So here we're picturing a beam of light running into a raindrop. And let's move it upwards to see how the light reacts at different points. What I want to focus on in particular is this ray over here that's being reflected off the back of the droplet. And we can watch how this reflected ray goes lower and lower, but starts to gradually slow down and sort of linger a little before finally turning around and beginning to move back upwards.

[00:33:10]So, if we now take a rolling average of all the light that's being reflected and refracted as our beam travels across the drop, we can get a clearer picture of how that lingering light causes some directions to appear much brighter than others. And for an observer on the ground, at least with the sun somewhere behind them, the directions we're most interested in are these two over here at an angle to the incoming light of around 51 and 42° respectively. All right. So, I've made a quick test function in the ray tracer just to visualize these angles so that we can make sure our camera and light are set up nicely to capture a rainbow. So, here we're just calculating the angle between the view direction and the direction of the sunlight and subtracting that from 42°, meaning that that 42° arc should appear as a black band in the sky. So, we have our camera pointed this way and let's then grab our sunlight over here and just adjust that angle until we can see the ring of the rainbow appear.

[00:34:05]All right, let's then have a look at the 51° angle as well. So, I'll just add that into the code here quickly.

[00:34:14]And now we can see a double rainbow all the way around.

[00:34:18]Let's maybe actually invert the color on this so that white shows where the brightest light should be. But now that we've located the rainbow, let's try to render it for real. Although, I am still going to cheat quite a lot admittedly.

[00:34:30]Like rather than modeling millions of individual raindrops, we can take a more probabilistic approach like we did with the fog earlier. So we'll calculate some distance into the scene that the ray might travel before hitting a raindrop based on the density of the rain and simply randomize the position of the raindrop somewhere in that vicinity. We can then just do a special test for a sphere collision at that point before going on to test all the other objects in the scene.

[00:34:55]Now, the way this is set up, each ray that we fire from the camera only sees this single random raindrop, meaning we won't have rays bouncing from one drop to the next as they would in real life.

[00:35:05]So, it's extremely simplified, obviously, but I'm hoping it's still sufficient to capture the main effect, I guess. Let's run it and find out.

[00:35:15]Okay, first attempt is just a big white haze really, but we maybe need to dial the sun settings in a bit. Just as a reminder from back in the first ray tracing episode, the sun is being sampled whenever a ray doesn't collide with anything in the scene. And it's done quite crudely by looking at the angle between the ray and direction towards the sun, raising that to some power based on the desired size of the sun, and lastly multiplying that by some intensity factor. So I'll try just reducing the size of the sun to make things more focused and up the intensity a bit to compensate for that. And then let's try rendering this again. And here's what we get.

[00:35:52]This time we can clearly see our two rainbow arcs, which is pretty exciting.

[00:35:56]And if we overlay our debug view quickly, we can see that they're exactly where we expected them to be. There is a rather depressing lack of color in this rainbow, though. And the one big ingredient missing here is that I've just set a constant refractive index of 1.3 for the raindrops, but that number in reality varies ever so slightly depending on the specific wavelength of light. So looking up some data, I came across this paper here comparing seawater and freshwater at different temperatures. And there's a nice little plot here where we can see the refractive index going from just below 1.33 at a 700 nm wavelength of light up to a little over 1.34 at 400 nm. So that roughly covers the wavelength range of visible light. We can also see the temperature dependence here, but it's pretty subtle and certainly not something we care about today. Instead, we'll just be grabbing this set of coefficients here, which can be simply plugged into this equation in order to approximate the refractive index at any given wavelength in the visible range.

[00:36:56]So, if we now run our little two-dimensional sweep over this raindrop again using the refractive index for red light and then for green light and finally for blue light, we can see that each one is just a tiny bit different.

[00:37:08]So, we can zoom in on this 42° angle here, and we can see how it's separated slightly into a red, green, and blue band. And looking up to our double rainbow angle, we can see the same thing has happened, just flipped upside down.

[00:37:21]Back in our shader, then I want to try assigning every ray we shoot from the camera a random wavelength from the visible spectrum and pass that into the ray tracing function so that it can look up the appropriate refractive index when it runs into a raindrop. Then instead of returning a color, the function now just gives back a single value for the amount of light coming in at that wavelength.

[00:37:41]And this means we'll need to make some alterations if we want colorful objects in the scene. But for now, with our empty scene, the intensity is determined purely by the sun, which currently doesn't care about wavelength at all, only the angle. In reality, the sun does actually emit some wavelengths more than others, and those are then further filtered through the Earth's atmosphere.

[00:38:00]But let's not worry about all that for today. What we do need to do though is convert the random wavelengths we're using into a color multiplied by the intensity that the ray tracer returns and some lose together in order to build up our image. I suspect it's better to actually defer converting these intensities to a color until right at the end of the render. But I just want to keep things simple for now since I'll certainly be exploring this stuff in much more detail in the future. So I'm going to do the color conversion right away using this data from the International Commission on Illumination. Essentially, it seems that back in 1931, they had a bunch of people try to just visually match a single wavelength of light to the color formed by three overlapping light sources that they could individually control the intensity of. And from those experiments, we get this big bunch of numbers that allow us to now transform our wavelength into a combination of red, green, and blue pixels that should perceptually be at least very similar to the pure wavelength itself. So, let's run this now on our depressingly white rainbow. And all of a sudden, it has burst into color.

[00:39:04]Let's zoom in to see it a bit better.

[00:39:06]And it looks like red, orange, and yellow are definitely there. Green and blue are at least somewhat present, though not super clear, I'd say. And there's certainly some sort of indigo violet towards the bottom. Turning up to our secondary hack, it is rather dim now with how spread out the light has become, but we can just barely make out the same colors again, but flipped around as expected.

[00:39:27]All right. So, it's a pretty crude and tacky implementation of course, but I'm pretty happy that we got this little rainbow out of it. All I'd like to do quickly to end things off for today is to try to integrate this rainbow into a simple scene. So, I've got a bunch of little dragons hanging out over here, but these have colors assigned to them, which our experimental spectral ray tracer no longer understands. That means rather than saying that a dragon is pink, for instance, we need to define how much light it reflects at different wavelengths. And to figure this out, I've hacked together an extremely crude tool where we're able to set a target color and then have it iteratively adjust the reflection spectrum to get closer and closer to that target. I'll talk more about this another time, but we can now use that information whenever a ray hits an object to figure out how much light should be bouncing off of it.

[00:40:15]So, returning to our test scene, let's see how that looks.

[00:40:19]Okay, the colors seem roughly all right, but let's compare this render to the result from our regular RGB rate tracer.

[00:40:26]And it looks like the spectral version is actually a little dim and has a sort of strange tint to it. Clearly, my code is a bit dodgy, but I reckon it's close enough for our quick test today. So, let's try combining some colorful glass dragons with our little rainbow for one final render.

[00:40:47]All right, I think that is a fitting end to our adventure for today. Although, as is usually the case, we've uncovered a bunch more to explore. So, the journey never ends. In particular, I really want to learn how to do this spectral stuff properly so that we can render all sorts of interesting effects in the future.

[00:41:03]But that's all for today. So, thanks for watching and until next time. Cheers.