Central Limit Theorem

There's a proof of the Central Limit Theorem which I am very fond of, which is not often seen in textbooks. It is a sort of renormalisation argument. In a way, it's not even a rigorous proof — often the way with renormalisation — but in conjunction with the more classical proofs it lends a powerful insight.

Without loss of generality, let's consider a whole bunch of identical, univariate variables $X_n$, each with zero mean. Thus we have quite trivially that a sum of $N$ of them will still have zero expectation, and a variance of $N$.

Now, rather than summing all of them at the same time, we do it in steps, and renormalise along the way to see what happens. So concretely, let $X$ have a distribution given by $f$, which is assumed to be sufficiently well-behaved for whatever we need to do. Then $X+X \sim f \star f$, where the multiplication is a convolution. This is our coarse-graining step, so we still need to re-scale, so that we get back a univariate distribution: $$f^\prime(x) = \sqrt 2 (f \star f)(\sqrt 2 x).$$ Convolutions are easiest to take in Fourier space: $$\widetilde{f^\prime}(k) = \left[ \tilde{f}(k/\sqrt{2}) \right]^2.$$ It is then fairly trivial to check that the univariate Gaussian $\widetilde{f^*}(k) = e^{-k^2/\sqrt{2}}$ is a fixed point.

We can view this step as a transform on the space of distributions, and so it makes sense to linearise about this fixed point and look at what happens to small perturbations: \begin{align*} \widetilde{f^*}(k) + \widetilde{g_n}(k) &\rightarrow \widetilde{f^*}(k) + 2 \widetilde{g_n}(k/\sqrt{2}) \widetilde{f^*}(k/\sqrt{2}) + \mathrm{h.o.t.} \\ &= \widetilde{f^*}(k) + \lambda_n \widetilde{g_n}(k) \end{align*} Which has solutions as $\widetilde{g_n}(k) = (ik)^n \widetilde{f^*}(k)$ with eigenvalues $\left|\lambda_n\right| = 2^{1-n/2}$; these correspond to mathematically meaningful perturbations if and only $n$ is an integer greater than 0, for reasons of convergence and normalisation. That still leaves $n=1$ or $n=2$ as being relevant and marginal; the former correspond to shifting the mean, the latter to changing the variance — and since those are not allowed by assumption, we find that the Gaussian is a stable fixed point.

Notice that this does not say anything about the size of basin of attraction, so if another fixed point existed it could cause finite perturbations to actually flow away. Indeed, this makes it not quite a proper proof. On the other hand, this procedure gives the actual rate of convergence to a Gaussian, something that the classical proofs do not give.