Gibbs v Boltzmann Entropies (Supposedly)

18Oct07

It’s easy to believe, if you imbibe ‘canonical’ statistical mechanics textbooks uncritically, that thermodynamic entropy, defined by $\Delta S = \int \frac{dQ}{T}$ , has been successfully ‘reduced’ to statistical mechanical entropy. The situation is more complicated than that, though. Part of the problem is that there are many different non-thermodynamic notions of what entropy is. There is information-theoretic entropy. There is the Gibbs entropy. And then Boltzmann himself actually formulated more than one definition of entropy (and they do not refer to the same thing).

In a 1964 paper,¹ E T Jaynes attempts to elucidate the difference between the Gibbs entropy and what he refers to as the “Boltzmann entropy”.²

Jaynes defines the “Gibbs H” as $H_G = \int W_N \log W_N d\tau$ , where $W_N(x_1, p_1; x_2, p_2; \cdots ; x_N, p_N; t)$ is the probability distribution function of the system in its full phase space and $d\tau = d^3x_1 \cdots d^3 p_N$ . So $W_N$ essentially measures the probability that the system is found in a certain range of microstates. $H_G$ is meant to be effectively a measure of the system’s entropy, and is not to be confused with the Hamiltonian of the system.

The “Boltzmann H” is defined as $H_B = N\int w_1 \log w_1 d\tau_1$ , where $w_1(x_1,p_1;t)$ is the single-particle probability density $w_1(x_1,p_1;t) = \int W_N d\tau_{-1}$ , $N$ is the number of particles in the system, and $d\tau_{-1} = d^3x_2 \cdots d^3 p_N$ . “Single-particle probability density” should be self-explanatory: it refers to the probability that a particle in the system will have a certain position and momentum. So already we can see a significant conceptual difference between the “Boltzmann H” and the “Gibbs H” — one considers the probability that the entire system is in a certain state; the other considers the probability that each particle of the system is in a certain state, and simply multiplies that by $N$ .

Jaynes goes on to show that the different Hs lead to different values for the change in entropy $\Delta S$ of a system between two states. Specifically, $\Delta S$ for the Boltzmann entropy does not include the influences of interparticle forces, whereas the Gibbs entropy does. So the Gibbs entropy is ‘more general’.

In section V of the paper, Jaynes purports to offer a dynamical “reason” for the second law of thermodynamics, which states that $\Delta S \geq 0$ . First he argues that $W$ in Boltzmann’s beloved equation $S = k\log W$ mst be equal to $e^{-H_G}$ for consistency with the Gibbs H, and that $W$ so defined “measures, in some sense, the phase volume of ‘reasonably probable’ microstates”. He then considers the canonical distribution on the phase volume $W_0$ of a system at t=0, where the system is known to have macroscopic parameters $(X_1(0), \cdots, X_n(0))$ . The canonical distribution will determine a “high-probability region” $R_0$ in $W_0$ — the region of phase space in which a system with the macroscopic parameters above is ‘most likely’ to reside. Jaynes then asks us to imagine performing an adiabatic transformation on the system. If we apply this transformation to all the systems in $R_0$ , then $R_0$ will be transformed into a new region $R_t$ . But Liouville’s Theorem says that any phase space distribution function of a classical system is constant along trajectories of the system. This implies that the $W_t=W_0$ : the phase volume occupied by the ensemble of systems defined by the macroscopic parameters $(X_1(0), \cdots, X_n(0))$ remains constant throughout its time evolution.

Now consider the new macroscopic state of the system: $(X_1(t), \cdots, X_n(t))$ . Let R’ be the region of phase space of the system that is compatible with the new macroscopic state, including points that are incompatible with the earlier evolution from $(X_1(0), \cdots, X_n(0))$ . The Gibbsian entropy is then $S'=k\log W'$ , where W’ is the phase volume of R’. In other words, this entropy is “a measure of all conceivable ways in which the final macrostate can be realised”.

This is where Jaynes plays his trump card. We have imagined an adiabatic transformation from $(X_1(0), \cdots, X_n(0))$ to $(X_1(t), \cdots, X_n(t))$ . Since the $X_i$ s are just macroscopic thermodynamic parameters, the transformation should be “experimentally reproducible”. So we should expect that every microstate compatible with the initial state, $(X_1(0), \cdots, X_n(0))$ should be able to evolve into a microstate compatible with the final state, $(X_1(t), \cdots, X_n(t))$ . This implies that $R_t$ , the set of points in phase space that evolved from $R_0$ under the transformation, is contained in R’. Jaynes then claims that this means $W_0 = W_t \leq W'$ , which is the second law of thermodynamics.

This part left me utterly flabbergasted. Not because it’s too difficult to understand, but because Jaynes seems to make some rather serious elementary mistakes. Take a look at the last step in his reasoning, for example. Recall that $R_t$ is a subset of $W_t$ (the “high probability” subset). Jaynes argues that $R_t \subset R'=W'$ . But since $W_t$ is ‘bigger’ than $R_t$ , we’re hardly entitled to infer that $W_t \leq W'$ !

I am mystified, too, by the mention of $R_0$ and $R_t$ at all. They don’t seem to play any role except for allowing Jaynes to make that error in the last step.

There are also some deeper errors in the reasoning process. One is that an adiabatic transformation by definition means $dQ=0$ , so $\Delta S = \int \frac{dQ}{T}=0$ . So how can $W' > W_0$ ? Surely the only possible outcome is that $W'=W_0$ , unless we want to abandon the thermodynamic definition of entropy. Oops, $\Delta S = \int \frac{dQ}{T}$ applys only if all states on the path of transformation are equilibrium states. So you can have $\Delta S > 0$ for an adiabatic transformation, provided that the transformation passes through non-equilibrium states.

Ok, maybe we can drop the ‘adiabatic’ part and consider the system evolving in isolation until it reaches the new macrostate in question, sticking to the Ws instead of the Rs. Then wouldn’t Jaynes’ reasoning hold? Because of the condition of experimental reproducibility, $W_t$ would still need to be contained in $W'$ , so $W_0 = W_t \leq W'$ . But if we don’t stipulate that the transformation must be adiabatic (and hence, presumably, controllable by humans), then we can’t ensure that given any initial macrostate $(X_1(0), \cdots, X_n(0))$ , the system will evolve into a final macrostate $(X_1(t), \cdots, X_n(t))$ after time t — we can’t rely on the transformation being “experimentally reproducible”. So the argument doesn’t work for general transformations either. In any case, even if the argument did work for adiabatic transformations, all it would show is that $W_0 \leq W'$ for adiabatic transformations, which hardly constitutes a general statement of the second law.

I really hope I’m wrong on this, because I’ve liked some of Jaynes’ other papers and had high hopes for this one. But it looks like Section V at least is a dud. Jaynes wants to lean on his supposed derivation of the second law to extend the notion of entropy to nonequilibrium states (since the thermodynamic definition covers only equilibrium states). But since his derivation fails, that’s a no-go as well. I have yet to find any papers that have pounced on this boo-boo of Jaynes’. Perhaps I’m wrong and it’s not a boo-boo at all. Or perhaps no one really cares, since it’s common knowledge amongst those who work on the foundations of statistical mechanics that there is as yet no satisfactory way to derive the time-asymmetric second law of thermodynamics from classical dynamics, so another way in which that derivation fails is not of much interest.

Jaynes can fob it off by pointing out that he did say at the start of Section V that “In the following we are not trying to give a rigorous mathematical description… We are trying rather to exhibit the basic intuitive reason for the second law.” But the failures of his ‘derivation’ are not due to insufficiently rigorous approximations. They’re major conceptual failures. So I don’t see how they can provide even a ‘basic intuitive reason’ for the second law.

Section VI of the Jaynes paper actually makes some interesting, substantive, not obviously flawed points, but this post is long enough, so I might talk about them another day.

1. E. T. Jaynes, Gibbs vs Boltzmann Entropies, American Journal of Physics, Vol. 33, No. 5. (1965), pp. 391-398.

2. The careful phrasing is to reflect that Boltzmann came up with more than one definition of entropy, that his definitions are mutually inconsistent, and finally, that Jaynes’ idea of the “Boltzmann entropy” might not even have been one intended by Boltzmann himself.³

3. See R. Swendsen, Statistical mechanics of colloids and Boltzmann’s definition of the entropy, American Journal of Physics, Vol. 74, No. 3. (2006), pp. 187-190.

Filed under: physics | Leave a Comment
Tags: statistical mechanics, thermodynamics

No Responses Yet to “Gibbs v Boltzmann Entropies (Supposedly)”

Feed for this Entry Trackback Address

No Comments Yet

Recent Comments

As if free will wasn… on Benjamin Libet, 1916-2007
a
Archives
Blogroll
Meta

Gibbs v Boltzmann Entropies (Supposedly)

No Responses Yet to “Gibbs v Boltzmann Entropies (Supposedly)”

Leave a Reply

Recent Comments

a

Archives

Blogroll

Meta