Backporch

Pillars · deep dive

Generating functions & moments

One function can carry an entire distribution. Differentiate it and the moments fall out; multiply two of them and you've added independent variables; take its logarithm and you get the cumulants. Push it to the imaginary axis and it becomes the Fourier transform of the density — the object that drives the central limit theorem. This deep dive builds that machine and wires it to the rest of the site.

Cross-references, so nothing is re-derived twice: the combinatorial generating-functions proof and the law of large numbers live in Proofs; the distribution catalog and the expected-value glossary live in the Library; Fourier itself is in change of basis.

1 · Package a distribution into one function

A generating function turns a whole sequence into a single object. For a counting sequence it's the ordinary generating function proved over in Proofs (Fibonacci → Binet's closed form); for a random variable there are three flavors:

G_{X} (z) = E [z^{X}] (PGF), M_{X} (t) = E [e^{tX}] (MGF), φ_{X} (t) = E [e^{i tX}] (char. fn).

The PGF is natural for counts, the MGF for moments, the characteristic function for limit theorems. The mean and variance of every distribution in the Library catalog are the first two moments these functions encode.

2 · Every shape number, by differentiation

Differentiate the MGF at the origin and the raw moments fall straight out — the MGF is the exponential generating function of the moments:

M_{X} (t) = n = 0 \sum \infty \frac{E [ X ^{n} ]}{n !} t^{n} ⟹ E [X^{n}] = M_{X}^{(n)} (0) .

But the raw moments are not yet the names you want. $M^{'} (0)$ is the mean — but $M^{''} (0) = E [X^{2}]$ is not the variance; you have to centre it, $Var (X) = M^{''} (0) - M^{'} (0)^{2}$ . The clean route is the cumulant generating function $K = lo g M$ : its derivatives $κ_{n} = K^{(n)} (0)$ are the named shape numbers — mean and variance are $κ_{1}, κ_{2}$ , then standardized skewness and kurtosis are $κ_{3}, κ_{4}$ , and the higher cumulants are the corrections Edgeworth and Cornish–Fisher expansions lean on. Hover a rung to see what it measures; then slide the shape toward the normal.

M (t) = E [e^{tX}] = E [1 + tX + \frac{( tX ) ^{2}}{2 !} + \frac{( tX ) ^{3}}{3 !} + \dots] = n \geq 0 \sum moment m_{n} E [X^{n}] \frac{t ^{n}}{n !}

1

m_{1} \frac{t ^{1}}{1 !}

m₁ = 2.00+

m_{2} \frac{t ^{2}}{2 !}

m₂ = 6.00+

m_{3} \frac{t ^{3}}{3 !}

m₃ = 24.0+

m_{4} \frac{t ^{4}}{4 !}

m₄ = 120+

m_{5} \frac{t ^{5}}{5 !}

m₅ = 720+

m_{6} \frac{t ^{6}}{6 !}

m₆ = 5,040+

\dots

The coefficient on $t^{n} / n!$ is the raw moment $m_{n} = M^{(n)} (0)$ . Pull it out, centre and standardize it, and it becomes a named number — the ladder.

Gamma(k) vs its normal twin ╌ — the gap is everything past variance.

1mean $μ = κ_{1}$ 2.00
2variance $σ^{2} = κ_{2}$ 2.00
3skewness $γ_{1} = κ_{3} / σ^{3}$ 1.414
4kurtosis $γ_{2} = κ_{4} / σ^{4}$ 3.000
55th cumulant $κ_{5} / σ^{5}$ 8.485
66th cumulant $κ_{6} / σ^{6}$ 30.000

shape k = 2.0

Read it across: the series hands you raw moments as coefficients; take $K = lo g M$ and its coefficients are the cumulants — the mean, variance, standardized skewness and kurtosis on the ladder. Hover any rung or term to light up its meaning on the shape. Then slide k up: the Gamma melts onto its normal twin and every number past variance falls to zero — the fingerprint of the normal, and the reason it is the one distribution with no shape left to name.

3 · Sums become products — and the bell appears

Here's why these functions earn their keep. For independent $X, Y$ :

M_{X + Y} (t) = E [e^{t (X + Y)}] = E [e^{tX}] E [e^{t Y}] = M_{X} (t) M_{Y} (t) .

Adding random variables convolves their densities — a painful integral — but it just multiplies their MGFs (the same convolution-to-product move Fourier provides in change of basis). Take the logarithm and the product becomes a sum, so cumulants add: $κ_{n} (X + Y) = κ_{n} (X) + κ_{n} (Y)$ . Stack $N$ copies and that single fact is the whole central limit theorem — the standardized shape numbers shrink and every distribution melts onto the same bell.

Xk2.0

Yμ1.0

M_{X} (t) =

1

m_{1}^{X} t

m_{2}^{X} \frac{t ^{2}}{2 !}

m_{3}^{X} \frac{t ^{3}}{3 !}

m_{4}^{X} \frac{t ^{4}}{4 !}

120+ ⋯

\times M_{Y} (t) =

1

m_{1}^{Y} t

m_{2}^{Y} \frac{t ^{2}}{2 !}

m_{3}^{Y} \frac{t ^{3}}{3 !}

m_{4}^{Y} \frac{t ^{4}}{4 !}

10+ ⋯

M_{X + Y} (t) =

1

m_{1} t

m_{2} \frac{t ^{2}}{2 !}

m_{3} \frac{t ^{3}}{3 !}

m_{4} \frac{t ^{4}}{4 !}

330+ ⋯

expand the coefficient of

m_{2} (X + Y) =

m_{0}^{X} m_{2}^{Y}

2 m_{1}^{X} m_{1}^{Y}

m_{2}^{X} m_{0}^{Y}

= 12

Read across: $M_{X + Y} = M_{X} M_{Y}$ , so adding the variables multiplies the series. But a product mixes orders — the coefficient of $t^{2} /2!$ picks up the cross terms (highlighted), not just $m_{2}^{X} + m_{2}^{Y}$ . That mixing is exactly why moments don't add. Take the $lo g$ and the product collapses into a clean sum — the cumulant series of §4, where every order adds on its own and nothing crosses between columns.

4 · Cumulants — the numbers that add

Take the logarithm of the MGF and you get the cumulant generating function $K = lo g M$ ; its coefficients are the cumulants $κ_{n} = K^{(n)} (0)$ . These are the named numbers from §2 — and they have a property raw moments lack: over independent sums they simply add. Build two distributions and watch the bars stack.

Xk2.0

Yλ3.0

K_{X} (t) =

κ_{1}^{X} t

κ_{2}^{X} \frac{t ^{2}}{2 !}

κ_{3}^{X} \frac{t ^{3}}{3 !}

κ_{4}^{X} \frac{t ^{4}}{4 !}

12+ ⋯

+ K_{Y} (t) =

κ_{1}^{Y} t

κ_{2}^{Y} \frac{t ^{2}}{2 !}

κ_{3}^{Y} \frac{t ^{3}}{3 !}

κ_{4}^{Y} \frac{t ^{4}}{4 !}

3+ ⋯

K_{X + Y} (t) =

κ_{1} t

2+3=5

κ_{2} \frac{t ^{2}}{2 !}

2+3=5

κ_{3} \frac{t ^{3}}{3 !}

4+3=7

κ_{4} \frac{t ^{4}}{4 !}

12+3=15+ ⋯

lo g (multiply (§3) M_{X} M_{Y}) = add (here) K_{X} + K_{Y} \Rightarrow κ_{n} (X + Y) = κ_{n} (X) + κ_{n} (Y)

Read down each column: one order, one color. Taking $lo g$ of the MGF turns the messy product from §3 into a clean sum of these two series — so the cumulants simply add, order by order, with nothing leaking between columns. Make a distribution Normal and its skew and kurt chips read 0: the bell adds nothing past variance. That order-by-order cleanliness is the whole reason cumulants — not moments — are the natural coordinates for sums.

5 · The characteristic function is the Fourier transform

Push the MGF to the imaginary axis and it becomes the Fourier transform of the density:

φ_{X} (t) = E [e^{i tX}] = \int_{- \infty}^{\infty} e^{i t x} f (x) d x .

Unlike the MGF, it always exists — $∣ φ_{X} (t) ∣ \leq 1$ for every $t$ , because $e^{i t x}$ rides the unit circle instead of racing off. It determines the law uniquely by inversion, and pointwise convergence of characteristic functions implies convergence in distribution (Lévy's continuity theorem) — which is exactly the march to the bell you just watched in section 3, and the real engine behind the law of large numbers and the CLT. This is the same inner-product / Fourier geometry as the L² deep dive, now applied to the density itself.

6 · What moments don't pin down

A caution that keeps everything honest: the first two moments are not the distribution. The Datasaurus dozen in the Judgement Lab all share the same mean, variance, and correlation while looking nothing alike — a dinosaur and a star with identical summary statistics.

Even the entire moment sequence can fail to determine a law (the lognormal is famously moment-indeterminate — different distributions, identical moments). That's exactly why the characteristic function matters: it always exists and always determines the distribution. Moments describe; the characteristic function decides.

7 · The calculator — every law, four ways

One engine for all of it. Pick a distribution and read off its four generating functions at once — the PGF, MGF, CGF, and characteristic function — as closed forms and as the color-coded power series of moments and cumulants. The menu is a set of legos; the counterexamples are in it on purpose — pick Cauchy and watch the MGF refuse to exist while the characteristic function stays perfectly smooth.

Poisson

λ

PGF

G (z) = e^{λ (z - 1)}

MGF

M (t) = e^{λ (e^{t} - 1)}

CGF

K (t) = λ (e^{t} - 1)

char. fn

φ (t) = e^{λ (e^{i t} - 1)}

PMF · the law itself

moments

m_{1}

m_{2}

m_{3}

m_{4}

309

m_{5}

1,866

m_{6}

12,351

cumulants

κ_{1}

κ_{2}

κ_{3}

κ_{4}

κ_{5}

κ_{6}

Four transforms, one law. The PGF marks counts, the MGF dispenses moments, its log — the CGF — gives the cumulants (the coordinates that add), and the characteristic function is the Fourier twin that always exists. Moving between them is change of basis; $φ (t) = M (i t)$ , $M (t) = G (e^{t})$ . Every cumulant equals λ — the cleanest fingerprint there is. The moments are the messy Bell polynomials of it.

8 · The calculator — combine two laws, X + Y

Now the operations. Pick X and Y and add them: convolving densities is hard, but an operation on the variables is algebra on their transforms — the MGF, PGF, and characteristic function multiply while the CGF adds, so the cumulants add order by order. When the sum lands back in a named family the calculator names it; pick two Cauchys and watch a law with no moments stay closed and stable — the warning that averaging can fail to tame a tail.

μ

σ

μ

σ

X + Y =

N (μ_{X} + μ_{Y}, σ_{X}^{2} + σ_{Y}^{2})

=

$N (3, 2)$

X

Y

$X + Y$ — convolved, wider & flatter

An operation on the variables is algebra on their transforms:

PGF— needs integer support

MGF

M_{X + Y} (t) = M_{X} (t) \cdot M_{Y} (t)

× multiply

CGF

K_{X + Y} (t) = K_{X} (t) + K_{Y} (t)

add ADD — the clean coordinates

char. fn

φ_{X + Y} (t) = φ_{X} (t) \cdot φ_{Y} (t)

× multiply — always works

κ^{X}

κ_{1}^{X}

κ_{2}^{X}

κ_{3}^{X}

κ_{4}^{X}

0+ ⋯

+ κ^{Y}

κ_{1}^{Y}

κ_{2}^{Y}

κ_{3}^{Y}

κ_{4}^{Y}

0+ ⋯

κ^{X + Y}

κ_{1}

1+2=3

κ_{2}

1+1=2

κ_{3}

0+0=0

κ_{4}

0+0=0+ ⋯

multiply M_{X + Y} = M_{X} M_{Y} l o g add K_{X + Y} = K_{X} + K_{Y} \Rightarrow κ_{n} (X + Y) = κ_{n} (X) + κ_{n} (Y)

Adding independent variables convolves their densities — hard. But it multiplies their MGFs (§3), and taking $lo g$ turns that product into a sum of CGFs (§4), so the cumulants add order by order while the moments mix. That is the calculator's one idea: an operation on the variables is algebra on the transforms, and each of the four transforms (§7) is one coordinate system for the same law. When the sum closes, it lands back in a named family; when X or Y is Cauchy, no MGF exists at all — but the characteristic function never flinches, and stability means averaging never tames it. The squared-distance L² metric on these variables is the next floor down.

9 · Partitions — one product, every way to bucket n

The combinatorial side, and the Ramanujan lego. A partition of $n$ is a way to drop $n$ balls into buckets; one infinite product is the generating function for every such grouping at once. Cycle through them — the balls regroup, the bucket sizes redraw as a treemap, and the coefficient of $q^{n}$ counts them. Label the balls and the same picture becomes what is a bucket? — set partitions, counted by Bell numbers.

n

6p(6) = 11 ways to bucket 6 identical balls

partition 1 / 11 ·

6 = 6

n balls grouped into buckets

bucket sizes, as a treemap

k \geq 1 \prod \frac{1}{1 - q ^{k}} = n \geq 0 \sum p (n) q^{n}

1 q^{0}

1 q^{1}

2 q^{2}

3 q^{3}

5 q^{4}

7 q^{5}

11 q^{6}

+ ⋯

Every way to drop $n$ identical balls into buckets is a partition — and the single product $\prod 1/ (1 - q^{k})$ is the generating function for all of them at once: the coefficient of $q^{6}$ is p(6) = 11. Hardy and Ramanujan found how fast that count grows; the pentagonal-number theorem hides inside the same product. Now label the balls and the buckets become sets — that is what is a bucket?, counted by the Bell number B(6) = 203 instead. Same picture, distinguishable balls. The same grouping is a TDA cluster, an equivalence class, a quotient — buckets all the way down.

Now Ramanujan's miracle, hiding in those counts: lay $p (n)$ in a grid and a whole column is divisible by 5 — and by 7, and 11. Watch the column light up:

p (5 n + 4) \equiv 0 (mod 5)

n \equiv 0

n \equiv 1

n \equiv 2

n \equiv 3

n \equiv 4

102

111

122

131

140

151

161

172

180

190

202

212

222

230

240

253

261

270

283

290

304

312

324

333

340

353

362

372

380

390

cell =

p (n) mod 5

· green = divisible by 5 · the highlighted column is all green

k \geq 1 \prod \frac{1}{1 - q ^{k}} = n \geq 0 \sum p (n) q^{n}

These are the same partition numbers $p (n)$ — and Ramanujan saw that the column $n \equiv 4$ is entirely divisible by 5: $p (5 n + 4) \equiv 0$ . Concretely p(4)=5, p(9)=30, p(14)=135, p(19)=490, p(24)=1,575 — every one a multiple of 5. It happens for 5, 7, and 11 only (nothing so clean for 13 or beyond). Why those three? The generating function is almost a modular form — $1/ η$ with $η = q^{1/24} \prod (1 - q^{n})$ — and the proof rides the Hecke operators mod $p$ you met in the Eigenbook: the partition function is a shadow of a modular form, and these congruences are that shadow's arithmetic. The same machinery, one floor down — from buckets of balls to $η$ to Frobenius.

And here is the engine that computes every one of those $p (n)$ — Euler's pentagonal number theorem. Multiply out $(1 - q) (1 - q^{2}) (1 - q^{3}) \dots$ and almost everything cancels; slide the factors in and watch the coefficients lock into a sparse $\pm 1$ pattern:

factors K6

k = 1 \prod 6 (1 - q^{k})

q^0+1q^1−1q^2−1q^30q^40q^5+1q^60q^7+2q^80q^9−1q^10−1q^11−1q^12−1q^130q^14+2q^150q^16+1q^170q^180q^19−1q^20−1q^21+1q^220q^230q^240q^250q^260

solid = locked (degree ≤ K, final) · faded = still cancelling · colored = survives, at a pentagonal exponent

k \geq 1 \prod (1 - q^{k}) = j = - \infty \sum \infty (- 1)^{j} q^{j (3 j - 1) /2} = 1 - q - q^{2} + q^{5} + q^{7} - q^{12} - \dots

The exponents that survive — 1, 2, 5, 7, 12, 15, 22, 26, … — are the generalized pentagonal numbers $\frac{j ( 3 j - 1 )}{2}$ , and every surviving coefficient is just $\pm 1$ . An infinite product collapsing to almost all zeros is the marvel. And it is not a curiosity: this product is exactly $1 / \sum p (n) q^{n}$ , so multiplying out and matching coefficients gives a recurrence for the partition numbers, $p (n) = p (n - 1) + p (n - 2) - p (n - 5) - p (n - 7) + p (n - 12) + \dots$ the very engine behind every $p (n)$ in the grid above — e.g. $p (6) = p (5) + p (4) - p (1) = 7 + 5 - 1 = 11$ , the same 11 that opens Ramanujan's $mod 11$ column. The same product, dressed with $q^{1/24}$ , is the modular form $η$ — the bridge to the Eigenbook.

10 · The Mellin bridge — from q-series to modular forms

The door out. The same coefficients $a_{n}$ that ride on powers of $q$ as a modular form ride on powers of $1/ n$ as a Dirichlet series — and the Mellin transform is the bridge between them. This is Hecke's correspondence: the modular law of $f$ becomes the functional equation of $L$ . Toggle Ramanujan's Δ (a cusp form — $L$ entire, symmetric about $s = 6$ ) against Eisenstein E₄ (whose $L = ζ (s) ζ (s - 3)$ has a pole at the abscissa $s = 4$ — where Bohr–Landau and the Laurent expansion live).

additive — the modular form

a_{1} q^{1}

a_{2} q^{2}

-24

a_{3} q^{3}

252

a_{4} q^{4}

-1,472

a_{5} q^{5}

4,830+

\dots

f (τ) = n \geq 1 \sum τ (n) q^{n}, q = e^{2 π i τ}

⟷

Mellin

multiplicative — the Dirichlet series

a_{1} 1^{- s}

a_{2} 2^{- s}

-24

a_{3} 3^{- s}

252

a_{4} 4^{- s}

-1,472

a_{5} 5^{- s}

4,830+

\dots

L (s) = n \geq 1 \sum \frac{τ ( n )}{n ^{s}}

Λ (s) = (2 π)^{- s} Γ (s) L (s) = \int_{0}^{\infty} f (i y) y^{s - 1} d y

Λ (s)

· symmetry axis s = k/2 = 6

s8.00

f (- 1/ τ) = (τ / i)^{k} f (τ) ⟹ Λ (s) = Λ (k - s), k = 12. L entire — no pole.

One set of numbers, τ(n), read two ways: stacked on powers of q it is a modular form (a generating function); stacked on powers of 1/n it is a Dirichlet series. The Mellin transform is the bridge, and the modular law of $f$ becomes the analytic law of $L$ . Because Δ is a cusp form, its $L$ is entire and perfectly symmetric about s = 6 — Hecke's functional equation, drawn. Square the spectrum of the Eigenbook and this is the analytic spine it climbs.

Study these — derivation cards

Reconstruct before you flip. The matching spaced-repetition deck lives in the Obsidian vault.

Prompt 1 · tap to reveal

Define the moment generating function and the characteristic function.

1 / 8