Thermodynamics is the study of heat, temperature, pressure, and related phenomena. Physicists have long considered it the physics area least likely to be overturned by future discoveries, in part because they understand it so well via "statistical mechanics." Alas, not only are we far from understanding thermodynamics, the situation is much worse than most everyone (including me until now) admits! In this post I'll try to make this scandal clear.

For an analogy, consider the intelligent design question: did "God" or a "random" process cause life to appear? To compute Bayesian probabilities here, we must multiply the prior chance of each option by the likelihood of life appearing given that option, and then renormalize. So all else equal, the less likely that life would arise randomly, the more likely God caused life.

Imagine that while considering ways life might arise randomly, we had trouble finding *any* scenario wherein a universe (not just a local region) randomly produced life with substantial probability. Then imagine someone proposed this solution: a new law of nature saying "life was sure to appear even though it seems unlikely." Would this solve the problem? Not in my book.

We are now in pretty much in this same situation "explaining" half of thermodynamics. What we have now are standard distributions (i.e., probability measures) over possible states of physical systems, distributions which do very well at predicting *future* system states. That is, if we condition these distributions on what we know about current system states, and then apply local physics dynamics to system states, we get excellent predictions about future states. We predict heat flows, temperatures, pressures, fluctuations, engines, refineries, etc., all with great precision. This seems a spectacular success.

BUT, this same approach seems spectacularly *wrong* when applied to predicting *past* states of physical systems. It gets wrong heat flows, temperatures, and pretty much everything; not just a little, but a lot wrong. For example, we might think we know about the past via our memories and records, but this standard approach says our records are far more likely to result from random fluctuations than to actually be records of what we think they are.

Physicists' and philosophers' standard response to this problem is much like invoking a "life appeared even though it seems unlikely" law: they invoke a "past hypothesis" saying the universe long ago had "very low entropy." Now when we clump physical states into "coarse" states roughly corresponding to what one might know about a system via crude observations, the "entropy" of each coarse state is the log of the weight that a standard distribution gives to that clump. So saying that early systems have "very low entropy" just says that the distributions we would usually use to successfully predict their futures are completely, totally, and almost maximally WRONG for predicting their pasts!

So we "resolve" the massive mistakes standard distributions make when applied to the past by adding a "law" saying basically, "what works well predicting the future makes near maximal mistakes when predicting the distant past." Which seems to me to basically say that this anomaly is about as bad as it could possibly be. Worse, this "past hypothesis" is ambiguous in several ways: it doesn't say exactly to what "early" space-times it applies, nor just how "near" maximally wrong standard distributions will be there, nor which of the many very wrong distributions apply.

Furthermore, it is not even clear that such a hypothesis achieves its intended end. We have many detailed formal calculations predicting future states given standard assumptions, but I'm aware of no formal calculations predicting past states given the usual machinery plus such a distant past hypothesis. And I've seen no calculations formally evaluating this hypothesis relative to other hypotheses. So the "past hypothesis" seems more of a vague hope than a reliable inference technique.

Only a tiny handful of physicists (and philosophers) are trying to explain this past hypothesis, i.e., looking for concrete assumptions that might imply it. And reviewing their efforts over the last few days I have to report: no one is even remotely close. Yet thermodynamics is usually taught as if this problem doesn't exist. Scandalous!

So Eliezer was wrong to say "The Second Law of Thermodynamics is a corollary of Liouville's Theorem"; the second law makes predictions about both future and past, and while future predictions are corollaries, past ones are not. And I was wrong to suggest that "at least one inflation origin ... implies at least one (and perhaps infinitely many) large regions of time-asymmetry like what we see around us." Sean Carroll was correct to respond:

In technical terms, I was fooled by the low dimensional state spaces commonly used to model inflation; unless strange physics fails to map states one-to-one across time, an initial inflation bubble must have as many possible states as any vast universe of eternally expanding bubbles that might follow from it.

For more, see this review article, this book, and this post and presentation by Carroll.

Even if the hypothesis of a low-entropy origin of the universe was proved false, this would not invalidate thermodynamics or the 2nd Law. So clearly it is incorrect to say that thermodynamics somehow "depends" on the Past Hypothesis or that it is somehow "scandalous" that the cosmological implications of the 2nd Law are not explained in textbooks.

Posted by: David | March 28, 2009 at 02:20 PM

It almost seems like people are iterating through all possible misunderstandings, suggesting I should have just ended the post with all possible disclaimers. :)

Phil, I said statistical mechanics was bad at predicting the past, not thermodynamics; it gets teakettles wrong too.

Scent, we know how to calculate the future, so calculating the past would be just as easy if the same approach applied.

James, Scent is right about records.

mjgeddes, tim is right; high entropy states can be very simply indicated.

Posted by: Robin Hanson | March 28, 2009 at 03:18 PM

Robin, this is false, weakly so in the first case[1], strongly so in the second. Please read what I write.

[1]Unless you have, for example, the definitive explanation on whether the universe is open or closed, etc. You are confusing local, gravity-free cases with global, non-gravity free cases. Iow, don't extrapolate from a gas enclosed inside a jar.

Posted by: ScentOfViolets | March 28, 2009 at 03:50 PM

Robin:

The low-entropy-past hypothesis is neither a full solution nor a restatement of our failure; it is a partial explanation. It is like starting with the question "Why is this room hot?" and getting the answer "because there is a thermostat connected to a heating device, which produces heat until the room is brought to a certain temperature". Sure, this explanation prompts the question "why is the thermostat set so high?", but that doesn't mean the explanation is just a useless restatement of the question.

In comparing the low-entropy-past hypothesis to the fictional "life appeared even though it seems unlikely" law, you ignore the substantial predictive power of the former.

Posted by: Jess Riedel | March 28, 2009 at 05:13 PM

You've got to be very careful how to set up the boundary conditions, even in a very simplified classical regime. For example, any collection of particles assembled in a finite volume of space, no matter how large, and allowed to expand outward for whatever length of time you care to specify into a vacuum of infinite space will be in a condition of 'low entropy' in comparison to later times. It's also not very difficult to define conditions so that the total energy of the universe is zero. Thus, a localized 'quantum fluctuation' in infinite space and time would under these conditions quite nicely start out in a condition of 'low' entropy and propagate forward in time into the infinite future in a very natural way.

I'm not endorsing this simplified model at all, btw. Just pointing out that reasoning by analogy to a more familiar situation can lead to apparently nonsensical results.

Posted by: ScentOfViolets | March 28, 2009 at 05:35 PM

@Tim Tyler: I follow that, but it doesn't matter: obviously the concisely-describable high entropy states are a tiny fraction of their total number, so it's *still* justified to assign a non-negligible prior to a low-entropy state if you're using Solomonoff induction.

Posted by: Paul Crowley | March 28, 2009 at 08:15 PM

Stuart, yes I recall Penrose's flat past hypothesis; is it still considered viable?Hard to tell. It is very speculative, and Penrose is reaching the end of his career, and some people are clearly humouring him... But there is some weak evidence supporting it, and it does appear to be testable.

And the mathematics appear correct - in the absence of the Ricci tensor, the slow, infinite, cold end of the universe is indistinguishable from the fast burning expansion of the big bang...

Posted by: Stuart Armstrong | March 30, 2009 at 05:12 AM