The Shannon-Hartley Formula Is a Statement About Gaussian Noise Only — Epoche C1
The formula $C = B \log_2\!\left(1 + S/N\right)$ gives the largest rate, in bits per second, at which information can be sent over a channel of bandwidth $B$ hertz carrying average signal power $S$ against average noise power $N$, with an error probability that can be driven as close to zero as desired. It is due to Claude Shannon (1948) and carries Ralph Hartley's name because Hartley (1928) had already argued that the information conveyed by a transmission is proportional to the logarithm of the number of distinguishable signal levels; Shannon's contribution was to say what "distinguishable" means when the corruption is random, and to prove that the resulting rate is not merely a good design target but an exact boundary. The formula is derived for a specific channel — additive white Gaussian noise — and this essay is about what changes when that model does not hold. The compressed version of this essay suggested that under non-Gaussian noise "the effective capacity might be higher or lower". That is the received wisdom and half of it is false. For additive noise of a given power, the Gaussian case is the worst case: any other noise distribution of the same power admits at least as high a capacity, and usually higher. The Shannon–Hartley formula is therefore not a ceiling that exotic channels might sneak past. It is a floor, and the interesting question is where the floor stops applying altogether. Deriving the formula, so that every factor has a reason The general expression for the capacity of a memoryless channel — one whose behaviour on each use is independent of the others — is $$C = \max_{p(x)} I(X;Y),$$ where $X$ is the transmitted symbol, $Y$ the received one, $p(x)$ the distribution the transmitter chooses over inputs, and $I(X;Y)$ the mutual information : the average reduction in uncertainty about $X$ produced by observing $Y$. Writing it as $I(X;Y) = h(Y) - h(Y \mid X)$, with $h$ the differential entropy of a continuous variable, makes the Gaussian case tractable. If $Y = X + Z$ with the noise $Z$ independent of $X$, then $h(Y \mid X) = h(Z)$, since once $X$ is known the only remaining randomness is the noise. So $$I(X;Y) = h(X+Z) - h(Z).$$ Two standard facts finish the calculation. First, a Gaussian variable of variance $\sigma^2$ has differential entropy $\tfrac12 \log_2(2\pi e \sigma^2)$ bits. Second — and this is the property the whole edifice rests on — among all distributions with a given variance, the Gaussian has the largest differential entropy. Hence with noise power $N$ we have $h(Z) = \tfrac12\log_2(2\pi e N)$ exactly, and $Y$, whose variance is $S+N$, satisfies $h(Y) \le \tfrac12\log_2\!\left(2\pi e (S+N)\right)$, with equality when $X$ is itself Gaussian. Subtracting, $$C_{\text{per use}} = \tfrac12 \log_2\!\left(1 + \frac{S}{N}\right) \ \text{bits per channel use}.$$ The factor $\tfrac12$ is not a convention; it comes from the variance sitting inside the logarithm in the Gaussian entropy, so that a ratio of powers becomes a ratio of squared amplitudes. The step to bits per second is where the bandwidth enters, and where the $\tfrac12$ disappears. Harry Nyquist's analysis of telegraph transmission (1928) established that a signal confined to a band of width $B$ hertz has only $2B$ degrees of freedom per second: it is determined by $2B$ samples, and no more can be sent independently. Multiplying, $$C = 2B \times \tfrac12 \log_2\!\left(1 + \frac{S}{N}\right) = B \log_2\!\left(1 + \frac{S}{N}\right) \ \text{bits per second}.$$ So the coefficient is $B$ rather than $2B$ or $B/2$ because Nyquist's factor of two exactly cancels the half from the Gaussian entropy. Nothing about the number 1 inside the logarithm is arbitrary either: it is the ratio $(S+N)/N$ written out. One consequence of this derivation is worth extracting, because it is the sharpest available refutation of the intuition that bandwidth buys rate without limit. White noise has a flat power spectral density $N_0$ watts per hertz, so the noise power in a band of width $B$ is $N = N_0 B$, and the noise grows as the band is widened. Using $\log_2(1+x) \to x/\ln 2$ as $x \to 0$, $$C_{\infty} = \lim_{B \to \infty} B \log_2\!\left(1 + \frac{S}{N_0 B}\right) = \frac{S}{N_0 \ln 2}.$$ Infinite bandwidth gives finite capacity. Writing $S = E_b C$, where $E_b$ is the energy expended per transmitted bit, and setting the rate equal to $C_\infty$, the $C$ cancels and leaves $E_b/N_0 = \ln 2 \approx 0.693$, which in decibels is $10\log_{10}(0.693) = -1.59$ dB. No system, of any bandwidth or complexity, can communicate reliably with less than that energy per bit relative to the noise density. The correction: Gaussian noise is the worst case The derivation above used the Gaussian assumption twice — once for $h(Z)$, once for the maximising input. It is natural to conclude, as the original text did, that with different noise the answer could go either way. It cannot, and the reason is a single inequality. Define the entropy power of a random variable $Z$ as $N_e = \tfrac{1}{2\pi e} 2^{2h(Z)}$: the variance a Gaussian would need in order to have the same entropy as $Z$. Because the Gaussian maximises entropy at fixed variance, $N_e \le N$ always, with equality precisely when $Z$ is Gaussian. The entropy power inequality, stated by Shannon in 1948 and first proved by Adriaan Stam in 1959, says that for independent $X$ and $Z$ $$2^{2h(X+Z)} \;\ge\; 2^{2h(X)} + 2^{2h(Z)}.$$ Its content is that entropy powers add at least as fast as variances do: summing independent variables spreads them out by at least as much as the Gaussian case would. Now take any additive noise $Z$ of power $N$, and let the transmitter send a Gaussian $X$ of power $S$ — not necessarily the optimal choice, which only strengthens the conclusion. Then $2^{2h(X)} = 2\pi e S$, and the inequality gives $$C \;\ge\; h(X+Z) - h(Z) \;\ge\; \tfrac12 \log_2 \frac{2\pi e S + 2\pi e N_e}{2\pi e N_e} \;=\; \tfrac12 \log_2\!\left(1 + \frac{S}{N_e}\right) \;\ge\; \tfrac1