Statistical Definition of Entropy | Boltzmann's Law Explained

[00:00:00]we've talked previously about the conceptual definition of entropy change as energy dispersal and in this video I want to shed more light on the actual nature of entropy by talking about the statistical definition of entropy itself before we dive in and start talking about the statistical definition of entropy we need to draw a distinction between the micro state of a thermodynamic system and its macro State the micro State consists of all of the positions and veloc ities the microscopic positions and velocities of all of the particles within the system so for example for an ideal gas if we were to sample or take a photo of an ideal gas system over time we would see the particles with different positions and different velocities pretty much in every snapshot one important thing to recognize about micro States ESP especially as you go on into more advanced coursework is that micro states are quantized according to Quant mechanics there are discrete energy levels that particles are allowed to access that leads to discrete velocities and even in a sense discrete positions accessible to the particles within a Quantum system that means that the number of possible micro States is finite since the number of states accessible is finite the macro State corresponds to the bulk thermodynamic macroscopic State functions for the system examples of these are pressure volume temperature internal energy and entropy for example and there are other examples that we've talked about as well such as enthalpy in the introductory videos we saw the idea that the macro State functions internal energy for example are equal to averages over the micro State microscopic energies are averaged to calculate the observed macro State internal energy U for example this idea is going to become important in a second the statistical definition of entropy deals with the question of how random are the possible micro States for a system given a particular macro state so how random are for example three possible micro states that a system could access little S1 little S2 and little S3 given a particular macro state capital or big S well let's think first about a hypothetical example where the micro State S1 has a probability of one while the probabilities of the second and third possible micro states are both zero well in this case we know exactly the micro state of the system essentially you can think of it like the particles of the system are frozen in space they're not moving so they have zero velocity and we know their positions exactly and so no other possible micro states can exist there's no Randomness in a system like this and so the entropy is z Z when there's zero Randomness in the possible micro States entropy is equal to zero let's look at a different probability distribution for the possible micro States let's imagine now that all of the probabilities were equal so the three micro States had probabilities of3 repeating each well in this case we can't really know which micro State the system is actually in since all three are equally probable and in this case the entropy is at a maximum and the range randomness of the micro States is at a maximum since we can't really know without sampling which micro State the system is actually in clearly the probability distribution over the micro States in other words what probability does each micro State have for all the possibilities figures deeply into the statistical definition of entropy and for the remainder of this video what what we want to do is flesh out this relationship mathematically stated bluntly how our entropy and the individual probabilities of specific micro States related in general there are a very very large number of possible accessible micro States for example for an ideal gas system and the number becomes explosive as you start adding particles and increasing volume but one thing we can say drawing an analogy from for example internal energy is that the observed macroscopic entropy has got to be awaited average over some function of these probabilities it remains to be determined what that function actually is but in writing this equation in the bottom right what we're doing is using for example the weighted average of internal energy as an analogy to find the average internal energy we weight each microscopic energy by its probability of existing the sum over all of those is equal to the average internal energy similarly with entropy what we can do is multiply some function of the probabilities we don't quite know what that is yet but you can think of that as the microscopic entropy the entropy per micro State we multiply that by the probability of that micro State existing and sum over all the possible micro states to get the average entropy that we would observe macroscopically so now we need to figure out the nature of this function f of the pi the function of the probabilities of the specific micro States existing that relates back entropy one place to start is this idea that the macroscopic entropy must be a weighted sum of this function of the probabilities each F of Pi weighted by the corresponding probability P sub I to flesh out this in more detail let's think about bolting two independent systems A and B together this is actually a common device in thermodynamics for learning more about thermodynamic State functions so for example let's say we took two ideal gases you might be getting sick of seeing the ideal gas by now I know I am and bolting the two of them together now ideal gases don't interact so the systems A and B are independent and the particles within the system are still independent in the combined system a plus b let's imagine that we had M accessible micro States in system a each with a set of probabilities P sub I so each of the M micro States has its own probability P sub I and in system B we had n possible micro States Each of which has a probability P subj how many total micro states are there for the combined system a plus b well we can't simply add the micro States and say M plus n because the two sets are independent and so for a particular micro state of system a let's call it I there are in possible micro states of system B and so what we're looking for here is not addition but multiplication the total number of micro states in the combined system is the product of the two not the sum what's the probability of a kind of composite micro state in which system a is in state I and system B is in state J well here again we need to multiply the two probabilities since the two systems are independent and the particles remain independent in the combined systems the probability of a particular micro state in the combined system in terms of the original probabilities is pi * PJ now what about the entropy of the combined system a plus b well for one thing we can use the equation in the top right now that we know the probability of a particular composite micro State we can just sum over all those possible combined micro States so we sum over the M for system a and the N for system B and do a weighted average of f of the probabilities so the probabilities are now P * PJ and so F of P * PJ times the probability P PJ added up over all the possible micro States is equal to the entropy this is just an incarnation of that equation in the top right where we've replaced p subi in that top right equation with pipj the new probability of the composite micro state in the combined system there's one other thing we should notice about the entropy of the combined system a plus b and it's that if we think back to our conceptual or intuitive Notions of entropy should be extensive that means when we bolt two systems together when when we combine system a and system B the resulting entropy of the combination should be the sum of the two entropies what we can do then is write the entropy of system a as the sum over the M possible micro States for that system Pi F of Pi and add to that the entropy of system B the sum over the n micro States p subj f of P subj so look at these two equations they're related to each other the entropy is equal to the sum over the two separate entropies and a sum over the composite micro States when you combine these equations with one another and do some mathematical manipulations you ultimately arrive at the idea that F of the product of two probabilities must be equal to the sum of f for those individual probabilities so F of Pi * PJ must be equal to F of pi plus F of PJ of course the only function for which this works is the natural log and so we now know then using that equation in the top right how entropy relates to the individual probabilities we can say s is equal to the sum over all the possible micro States the probability of that micro State times the natural log of the probability of that micro state in practice this is multiplied by a constant boltzman's constant and it's negated since the probabilities will be less than one and the natural logs will be negative the negation ensures that entropy remains positive so this is boltzman's statistical definition of entropy just reproduced from the last slot slide and it says that entropy is equal to negative boltzman's constant times the sum over all the possible capital N micro States now the probability of that micro State times the natural log of the probability of that micro state and this is a pretty neat concept actually you can relate boltzman's statistical definition back to thermodynamic quantities like heat and temperature and figure out some pretty interesting things about spontaneous processes for the time being I want to explore in more detail this second Factor this sum over all the possible micro States Pi natural log of Pi what is this exactly what does it mean and can we recast this equation in a somewhat more intuitive form in particular it's a little bit problematic to get for example the probability distribution over all the possible micro States when there are millions upon billions upon billions of possible micro states that probability distribution becomes simply unwieldy to deal with especially if you're talking about a quantized or discrete distribution you're talking about billions upon billions upon billions of numbers well one thing we can do to simplify this distribution is to move the pi Factor inside of the natural log moving P subi inside the natural log puts it as an exponent on Pi so we've got the sum over all the possible micro States the natural log of P sub I to the power of P sub I now if we blow this sum up and write it as natural log of pi to the pi for each term in the sum we get the natural log of P1 to the P1 power natural log of P2 to the P2 power plus the natural log of P3 to the P3 power power etc etc etc and sum of a set of natural logs is equal to the natural log of the product right and so we can pile all those terms into a single natural logarithm and say that this sum is equal to the natural logarithm of P1 to the P1 power * P2 to the P2 power * P3 to the P3 power now let's do something a little funky and plug this back in to the original definition of entropy but invert the argument of the natural logarithm such that a negative sign pops out front doing this the negative sign in the original equation is removed since we're multiplying a negative times a negative and the argument of the natural logarithm is inverted so we get the entropy is equal to boltzman's constant times the natural log of one over now P1 to the P1 power P2 to the P2 power etc etc etc all the way to the last micro State P subn to the P subn power this argument of the natural logarithm is naturally if you look at it going to be a very very large number if we have a lot of accessible micro States each of the probabilities is going to be quite low and this number is going to be very very large this is a number called W and it corresponds to the number of distinct micro states that lead to a particular or given macro State this is a more intuitive number to work with and I hope to demonstrate this to you with a quick example really quickly though before we do that that example notice that we can now rewrite boltzman's definition of entropy in the top left as s is equal to positive boltzman's constant times the natural log of w by thinking statistically and thinking in terms of combinatorics we can often write down a number for w or at least relative W values for two situations for example an initial and a final state so consider the following scenario you've got a box with six compartments in it and you start out with an ideal gas confined to one of those compartments with the volume v all the compartments have equal volume and so imagine for example you removed the inner walls so that the gas molecules were free to roam the entirety of the container and you had a situation where when you expanded the gas to 6V the 12 particles count them there are 12 particles in that ideal gas spread themselves out so that they're two particles in each sixth of the Box what's W for each of these situations well to figure out W you can think about the probability of a particular particle existing within a particular box in the case of the initial state with volume v there's really only one way to prepare this micro State we take all 12 particles of the ideal gas and throw them into the same box the number of ways to do this is 12 things taken 12 at a time or 12 C12 which is equal to 1 going back to our analogy from before this situation completely lacks Randomness we can identify with certainty that all of the particles are in box one therefore it makes sense that the entropy should be equal to zero and indeed it comes out that the entropy is equal to zero when we take the natural log of w equal to 1 right what happens when we allow the gas to expand to a volume of 6V well now what we can do is Imagine preparing this micro state by throwing two particles down into each box the number of ways to do this is much much larger than one because we start by taking for example two of the particles from the 12 and throwing them in the first box and there are 12 C2 12 things taken two at a time ways to do that we're then left with 10 and there are 10 C2 ways to throw two particles from the remaining 10 into the next box we multiply that by the 12 C2 since these two events throwing particles in box one and throwing particles in box two are independent and we can continue going with this so then we have eight particles left so for box three 8 C2 for box four 6 C2 box five 4 C2 and finally for the last last box 2 C2 and with only two particles left there's only one way to throw the remaining two particles into that last box but nonetheless if you go off and calculate for example 12 C2 and 10 C2 and hc2 you'll see that the product of all of these is going to blow up to be an enormous enormous number number just like we saw for the case of the spontaneously mixing gases a gas expanding like this corresponds to a very large change in entropy it is worth noting a couple of things though first of all the natural log of w is related to S so w can grow exponentially and S will grow only linearly and the other thing is that boltzman's constant is quite small so very very large changes in W can correspond to relatively small changes in s in for example jewles per Kelvin nonetheless this equation s is equal to KB natural log of w Really captures the statistical definition of entropy at its Essence the definition says that the more micro states that correspond to a particular macro State the more different ways there are to prepare a particular macro State the higher is the entropy of that macro State and in this example what we're seeing is there are simply more micro States associated with two particles in each hypothetical box than there are with all 12 particles in a single box that's why expanding the gas from V to 6V corresponds to an increase in s and a massive increase in W the number of possible micro States