The Practical Realities of Finite Blocklength Communication — Epoche C2
The design problem that capacity does not answer A 32-byte control message that must be delivered within one millisecond and may fail no more often than once in a hundred thousand attempts — the reliability and latency target that the Third Generation Partnership Project set for ultra-reliable low-latency communication in its Release 14 study of requirements for next-generation radio access — cannot be dimensioned by consulting Shannon's capacity formula. The obstacle is not that the formula is an approximation waiting to be refined. It is that the theorem which gives the formula its meaning makes a promise about a limit that a system of this kind never takes. What follows sets out what that theorem asserts, what replaces it when the block is short, and what the replacement costs in rate, in numbers that can be checked line by line. The belief under examination is that channel capacity, written $C$, is the rate any practical system could in principle reach, so that the gap between a deployed system and $C$ measures how much cleverness the designers still owe. For long blocks that is a fair reading, and it has served the field well for seventy years. For a 32-byte packet it is not merely optimistic; it misstates the problem, because at that length the binding constraint is not the average noise level but the fluctuation of the noise about its average over a small number of channel uses. What Shannon's theorem promises, and in what units Shannon's 1948 coding theorem is a statement about a sequence of codes, not about any one code. For a discrete memoryless channel of capacity $C$ and any rate $R \lt C$, there exists a sequence of block codes of length $n = 1, 2, \dots$ at rate $R$ whose probability of decoding error tends to zero as $n$ grows. The converse direction was sharpened by Wolfowitz in 1957 into what is now called the strong converse: for any fixed $R \gt C$, the error probability of the best length-$n$ code tends not merely away from zero but all the way to one. Taken together these two results say that, in the limit of unbounded blocklength, capacity is a sharp threshold — below it reliable communication is free, above it impossible. Everything a designer wants to know is contained in how that threshold behaves before the limit is reached, and Shannon's theorem is silent on exactly that point. A units confusion needs clearing before any arithmetic is possible, because the essay's own earlier formulation contained it. The familiar expression $C = B \log_2(1 + S/N)$, where $B$ is the bandwidth in hertz, $S$ the average received signal power and $N$ the noise power in that band, is Shannon's Theorem 17 and is measured in bits per second. The finite-blocklength theory counts channel uses, not seconds, and its quantities are per channel use. For a real-valued additive white Gaussian noise channel with signal-to-noise ratio $\rho$ — one real symbol transmitted, one real symbol received, corrupted by an independent Gaussian of variance $1/\rho$ times the signal variance — the capacity is $$ C(\rho) = \tfrac{1}{2}\log_2(1 + \rho) \quad \text{bits per channel use}. $$ The two expressions are the same statement counted differently: a strictly band-limited channel of bandwidth $B$ supports $2B$ real channel uses per second, and $2B \cdot \tfrac12 \log_2(1+\rho) = B\log_2(1+\rho)$. All the numbers below are per channel use, so that they can be multiplied by a blocklength $n$ and read as bits per packet. Where the asymptotics come from: the information density and its variance The finite-blocklength refinement is not a patch on Shannon's proof; it comes from noticing what quantity his proof averages. Fix an input distribution $P_X$ and let $P_{Y|X}$ be the channel. The information density of a particular input–output pair is $$ i(x; y) = \log_2 \frac{P_{Y|X}(y \mid x)}{P_Y(y)}, $$ the log-likelihood ratio between the hypothesis that $y$ was produced by the transmitted $x$ and the hypothesis that $y$ was drawn from the output distribution $P_Y$ that the code induces. Its mean under the capacity-achieving input is the mutual information, and at the optimising $P_X$ that mean is the capacity: $C = \mathbb{E}[i(X;Y)]$. Over $n$ independent uses the total information density is a sum of $n$ independent, identically distributed terms. Shannon's argument needs only the law of large numbers applied to that sum. The next order of accuracy is obtained by applying the central limit theorem instead, and for that one needs the second moment. The channel dispersion is exactly that second moment: $$ V = \operatorname{Var}\big[i(X;Y)\big] \quad \text{bits}^2 \text{ per channel use}, $$ evaluated under the capacity-achieving input distribution. Dispersion is therefore not a vague "measure of the channel's variability" but a variance with definite units, and it is what converts a limit theorem into an estimate at finite $n$. Polyanskiy, Poor and Verdú (2010) turned this observation into two-sided bounds. Their converse — the meta-converse — shows that any code with $M$ codewords and error probability $\epsilon$ yields a test that distinguishes $P_{Y|X=x}$ from an auxiliary output distribution $Q_Y$ with prescribed error levels, so that the best achievable $M$ is bounded by the optimal Neyman–Pearson trade-off for that binary hypothesis test. Their achievability bounds — the dependence-testing bound and the random-coding union bound — go the other way, showing that a random code decoded by thresholding the information density does at least as well as a computable expression. Both sides reduce to tail probabilities of the same sum of $n$ i.i.d. information densities; applying the Berry–Esseen theorem to that sum, which bounds the deviation of the sum's distribution from the Gaussian by a constant times the third absolute moment divided by $\sqrt{n}$, closes the two sides to within $O(\log n)$ in the logarithm of the code size. Writing $M^*(n,\epsilon)$ for the largest number of codewords a length-$n$ code can