The Logarithmic Price of a Longer Weather Forecast — Epoche C1
Operational weather forecasts lose most of their skill somewhere between one and two weeks ahead, and have done so for as long as they have been issued. The explanation usually offered is technological — the satellites were too sparse, the computer too slow, the model too coarse — and behind it sits a promise: that better instruments and faster machines will eventually deliver forecasts of any length we like. This essay examines that promise quantitatively and shows it cannot be kept. The obstacle is a single logarithm, and a second and harsher obstacle may lie behind it. Both were identified by Edward Lorenz, the first in 1963 and the second in 1969. Determinism is not predictability The common belief runs two different properties together. A system is deterministic if its present state fixes its entire future: start it twice from exactly the same state and it does exactly the same thing, with no randomness anywhere. A system is predictable only if a useful stretch of that future can be computed from data we actually possess. Real data always carry a finite error — a thermometer reads to a tenth of a degree, a radiosonde balloon drifts off station, and vast regions of ocean go unmeasured. The question that matters is not whether the equations are deterministic but what the dynamics do to that error. What Lorenz actually did The answer arrived by accident in 1961. Lorenz was running a twelve-variable numerical weather model on a Royal McBee LGP-30, a desk-sized valve computer, and wanted to repeat part of a run. He restarted it from numbers copied off the printout, which showed three decimal places where the machine held six — a discrepancy of roughly one part in a few thousand. The new run tracked the old one for a while, then diverged into entirely different weather. To establish that this was dynamics rather than an artefact of his particular model, he reduced the problem to the smallest system that could show it. His 1963 paper takes a severe truncation of the equations for Rayleigh–Bénard convection — a fluid layer heated from below — retaining just three Fourier amplitudes: $$\dot{x} = \sigma(y - x), \qquad \dot{y} = x(\rho - z) - y, \qquad \dot{z} = xy - \beta z .$$ The variables are not positions of anything. Here $x$ measures the intensity of the convective overturning, $y$ the temperature difference between rising and sinking currents, and $z$ the departure of the vertical temperature profile from a straight line. The parameters are the Prandtl number $\sigma$, the Rayleigh number expressed as a multiple of its critical value $\rho$, and a geometric factor $\beta$ fixed by the aspect ratio of the convection cells; Lorenz took $\sigma = 10$, $\rho = 28$, $\beta = 8/3$. Nothing in these equations is random, and the only nonlinearities are the two products $xz$ and $xy$. Solutions that begin arbitrarily close together nevertheless separate, on average, exponentially fast. That is what the word chaos names. The system remains perfectly deterministic; it simply refuses to remain predictable. The growth law, and why it is a theorem rather than a definition Write $\varepsilon(t)$ for the error of a forecast at time $t$ — the distance between the true state and the computed one, measured as a fraction of the size of a typical weather fluctuation. While the error is small, it grows as $$\varepsilon(t) \approx \varepsilon_0\, e^{\lambda t},$$ with $\varepsilon_0$ the initial error and $\lambda$ the largest Lyapunov exponent, the mean exponential rate at which neighbouring trajectories separate. It is tempting to call this equation merely the definition of $\lambda$, but that understates what is being assumed. A small perturbation $\boldsymbol{\delta}$ about a trajectory obeys the linearised equation $\dot{\boldsymbol{\delta}} = J(t)\,\boldsymbol{\delta}$, where $J$ is the Jacobian matrix of the vector field evaluated along the trajectory. Because $J$ changes as the trajectory moves, the solution is an ordered product of many different matrices, and there is no elementary reason why the growth of such a product should settle down to a clean exponential rate at all, nor why that rate should be the same for almost every starting point. That it does is the content of Oseledets' multiplicative ergodic theorem of 1968, which states that for a dynamical system preserving an invariant probability measure, the limit $$\lambda = \lim_{t\to\infty} \frac{1}{t}\,\ln\frac{\lVert \boldsymbol{\delta}(t)\rVert}{\lVert\boldsymbol{\delta}(0)\rVert}$$ exists for almost every initial state, and takes the largest of a finite set of values for all perturbation directions except those in a lower-dimensional subspace. Since a random observational error will not lie in that exceptional subspace, the largest exponent is the one that governs a forecast. For the Lorenz system at the parameters above, numerical computation of this limit gives $\lambda \approx 0.91$ per unit of the model's dimensionless time — a number about that model, not about the atmosphere, and one to keep separate from the atmospheric figures used below. A forecast stops being useful once its error is comparable with the signal, $\varepsilon(T) \approx 1$. Solving the growth law for that moment gives the predictability horizon $$T \approx \frac{1}{\lambda}\,\ln\frac{1}{\varepsilon_0},$$ which involves nothing deeper than inverting an exponential. All the pessimism lives in the logarithm. What a thousandfold improvement buys Now put in atmospheric numbers, taken from forecast practice rather than from the Lorenz model. Suppose the e-folding time $1/\lambda$ — the time over which an error grows by a factor $e \approx 2.718$ — is two days. The corresponding doubling time is $\lambda^{-1}\ln 2 \approx 1.4$ days, the order of magnitude reported for large-scale errors in modern forecast systems. An initial relative error of $\varepsilon_0 = 10^{-3}$ then gives $T = 2\ln 10^{3} \approx 13.8$ days. Improve the entire observing system a thousandfold, to