The Nuance of Product Expectations — Epoche C1
The Illusion of Simple Products In probability and statistics, few relationships appear as intuitively appealing yet are as frequently misunderstood as the expectation of a product of random variables. Recall first what the objects are. A random variable $X$ is a numerical quantity whose value is determined by chance — the outcome of a die, tomorrow's temperature — and its expectation $E[X]$ is its probability-weighted average: for a discrete variable, $E[X] = \sum_x x\, P(X = x)$, the sum of each possible value times the probability of that value; for a continuous variable the sum becomes an integral against the probability density. Many students, and some seasoned practitioners, operate under the assumption that for any two random variables $X$ and $Y$, the expectation of their product satisfies $E[XY] = E[X]E[Y]$. This seemingly straightforward equality is in fact a special case, valid only under a specific and crucial condition — independence — and the purpose of this essay is to show exactly where the general rule comes from, what fills the gap when independence fails, and how the gap is exploited or ignored in practice. Why linearity is universal — and why it misleads The genesis of the misconception is a correct theorem. One of the first properties taught about expectation is linearity: for any constants $a, b$ and any random variables $X, Y$, $$ E[aX + bY] = aE[X] + bE[Y], $$ with no assumption whatsoever about how $X$ and $Y$ are related. It pays to see why the assumption is not needed. Working with discrete variables and their joint distribution $p(x,y) = P(X = x, Y = y)$ — the table giving the probability of every pair of values simultaneously — we compute $$ E[aX + bY] = \sum_{x,y} (ax + by)\, p(x,y) = a\sum_{x,y} x\, p(x,y) + b\sum_{x,y} y\, p(x,y) = aE[X] + bE[Y], $$ where the last step uses the fact that summing $p(x,y)$ over all $y$ leaves the distribution of $X$ alone. The manipulation succeeds because $ax + by$ is itself a linear expression: the sum splits term by term, and the joint behaviour of $X$ and $Y$ — how their values pair up — never gets a chance to matter. The robustness of this result tempts one to an erroneous generalisation: if sums pass through $E$ unconditionally, why not products? But $xy$ is not a linear expression in $(x,y)$, and $\sum_{x,y} xy\, p(x,y)$ does not split into separate sums: each term couples a value of $X$ with a value of $Y$ through the joint probability of that pair. The product expectation is precisely a measurement of how values pair up, which is why it cannot, in general, be reconstructed from the two marginal averages. Unpacking the Covariance Term The correct general relationship is $$ E[XY] = E[X]E[Y] + \text{Cov}(X, Y), $$ where $\text{Cov}(X,Y)$, the covariance of $X$ and $Y$, is defined as $E[(X - E[X])(Y - E[Y])]$: the average product of the two variables' deviations from their own means. This identity is not a deep theorem but a two-line consequence of linearity itself, and deriving it removes all mystery. Write $\mu_X = E[X]$ and $\mu_Y = E[Y]$; these are fixed numbers, not random. Expanding the product of deviations, $$ (X - \mu_X)(Y - \mu_Y) = XY - \mu_Y X - \mu_X Y + \mu_X \mu_Y , $$ and taking expectations of both sides — legitimate term by term, because linearity holds universally, constants $\mu_X, \mu_Y$ passing outside $E$ — gives $$ \text{Cov}(X,Y) = E[XY] - \mu_Y \mu_X - \mu_X \mu_Y + \mu_X \mu_Y = E[XY] - E[X]E[Y]. $$ Rearranged, that is the boxed relationship above. The covariance is therefore, by construction, the exact error committed by the naive product rule : positive when $X$ and $Y$ tend to sit on the same side of their means (both high together, both low together), negative when one tends to be high while the other is low, zero when the deviations balance out on average. One special case shows how catastrophic the naive rule is if taken literally. Set $Y = X$. Then $\text{Cov}(X,X) = E[X^2] - (E[X])^2$, which is precisely the variance $\text{Var}(X)$, the standard measure of a variable's spread. Anyone who believes $E[XY] = E[X]E[Y]$ for all pairs must in particular believe $E[X^2] = (E[X])^2$, i.e. that every random variable has variance zero — that nothing random exists. A single fair coin refutes it: for $X$ equal to 1 on heads and 0 on tails with $P(\text{heads}) = p$, we have $X^2 = X$, so $E[X^2] = p$ while $(E[X])^2 = p^2$, and the discrepancy $p - p^2 = p(1-p)$ is exactly the coin's variance. Independence, and why it factorises Independence is the condition under which the covariance term is guaranteed to vanish. Two random variables are independent when the joint distribution factorises into the product of the marginals: $p(x,y) = p_X(x)\, p_Y(y)$ for all pairs $(x,y)$ (in general, $P(X \in A,\, Y \in B) = P(X \in A)P(Y \in B)$ for all events). Informally: knowing the value of $X$ provides no information about the value of $Y$, and vice versa. Under this condition the coupling that blocked the product computation dissolves: $$ E[XY] = \sum_{x,y} xy\, p_X(x)\, p_Y(y) = \Big(\sum_x x\, p_X(x)\Big)\Big(\sum_y y\, p_Y(y)\Big) = E[X]E[Y], $$ the double sum factorising because each summand is now a product of a term depending only on $x$ and a term depending only on $y$. (A technical proviso, standard in the measure-theoretic treatments such as Durrett's: the manipulation presumes the expectations involved are finite. For square-integrable variables the Cauchy–Schwarz inequality, $|E[XY]| \le \sqrt{E[X^2]\,E[Y^2]}$, guarantees that $E[XY]$ is well defined; it also shows that the correlation coefficient $\rho = \text{Cov}(X,Y)/(\sigma_X \sigma_Y)$, the covariance rescaled by the two standard deviations to a dimensionless number, always lies between $-1$ and $1$.) When variables are dependent, nothing protects the factorisation, and in fields where variables are rarely independent — econometrics, financial modelling, signal processing — the covariance term is not a nuisance but the object of study. A w