Change of basis & variable

The deepest move in mathematics and inference is also the quietest: re-express the same object in coordinates that make the question easy. The reality doesn't move — the frame does. Here's the law that makes it rigorous, a catalog of the standard moves, the derivations of $t$ and $F$ from $χ^{2}$ that fall out of symmetry (with a live forge), Fourier series as the canonical change of basis (with a live builder), and the real question — how do you know which leap to make?

0 · Logarithms — the inverse, the integral, the equation

Before any change of coordinates, the one that turns multiplication into addition. A logarithm is three things at once: the inverse of $e^{x}$ , the running area under $1/ t$ , and the unique continuous function with $ln (ab) = ln a + ln b$ . That last equation is the quiet engine of this whole deep dive — and of the cumulants a few pillars over. Drag both panels.

1/ t

area =

ln x

ln x = \int_{1}^{x} \frac{d t}{t}

= +1.001 (area past 1)

x = 2.72

l n a

l n b

l n (ab)

ln (a b) = l n a + l n b

· 0.69 + 1.10 = 1.79

a2.0b3.0

ln x = \int_{1}^{x} \frac{d t}{t}, \frac{d}{d x} ln x = \frac{1}{x}, e^{l n x} = x, ln (ab) = ln a + ln b

The reciprocal is the rate; its running area from 1 is $ln x$ — positive past 1, negative before it, exactly $0$ at 1. And because area concatenates, ln a and ln b add to ln(ab): multiplication becomes addition. That one equation is the engine three floors up — it's why cumulants add (log of a product of MGFs), why log-scale turns exponential growth into a straight line, and why the cleanest change of basis is often just the move to additive coordinates.

1 · One move, two flavors

A change of basis rewrites a vector or operator in new coordinates $v^{'} = P^{- 1} v, A^{'} = P^{- 1} A P$ ; a change of variable does the same for a function or an integral, $y = g (x)$ . Same idea: leave the object alone, choose a frame that simplifies the question. This deep dive extends the change-of-basis pillar — and reuses the whitening and eigenbasis machinery from the L² deep dive.

2 · The law that keeps it honest — the Jacobian

When you push a density through a map, probability mass must be conserved, so you pay the local volume change — the Jacobian. For an invertible $y = g (x)$ :

f_{Y} (y) = f_{X} (g^{- 1} (y)) det J_{g^{- 1}} (y) .

In one dimension that determinant is just $∣ d g^{- 1} / d y ∣$ . Forget the Jacobian and your new density won't integrate to one — the single most common change-of-variable mistake.

3 · A catalog of moves

The standard repertoire — reach for the row whose "when" matches your problem.

Move	When	Gain	Cost	Example
Eigenbasis / diagonalization	a linear map acts repeatedly	decouples into per-axis scaling	needs diagonalizability; axes may be complex	Aⁿ, Markov stationarity, PCA
Whitening / standardization	correlated, differently-scaled features	uncorrelated, unit-variance coords	loses units; needs Σ	Mahalanobis, pre-ridge scaling
PCA (truncated eigenbasis)	high-dimensional, redundant data	keep variance, drop noise axes	linear; axes can be uninterpretable	compression, denoising
Fourier / frequency	shift-invariant operators, convolution	convolution → product; ∂ₓ → ×(in)	global basis, poor for local features; Gibbs	signal processing, PDEs, char. functions
Generating function / MGF	sequences; sums of independents	convolution → product; moments by ∂	may not converge; needs inversion	sums of RVs, combinatorics
Log / Box–Cox	multiplicative or skewed data	multiplicative → additive; stabilizes variance	undefined at ≤ 0; remeans the data	lognormal, growth, GLMs
Polar / spherical	rotational symmetry	integrals separate; tidy Jacobian	only when the symmetry is there	Gaussian integral, radial PDEs
u-substitution / pushforward	an integral with hidden structure	reduces to a standard form	must track Jacobian & limits	change of variables in densities
Change of measure	the convenient law isn't the target	reweight to the target	needs the likelihood ratio; variance can blow up	importance sampling, Girsanov

Three of these, up close.

Diagonalization is the purest linear change of basis: in the eigenbasis a matrix is nothing but per-axis stretching, so $A^{n} = Q Λ^{n} Q^{- 1}$ and a hard repeated action becomes scalar powers. It is the same fact behind Markov stationarity (a left eigenvector with eigenvalue 1) and PCA (the eigenbasis of $Σ$ ).

Fourier earns its place because it diagonalizes every shift-invariant operator at once: differentiation becomes multiplication by $in$ , and convolution becomes a plain product, $f * g = \hat{f} \overset{g}{^}$ . That is why filtering, PDEs, and the characteristic function all live in the frequency frame.

Polar coordinates are the textbook "leap": the Gaussian integral is intractable in $x$ , but squaring it exposes a rotational symmetry, and in polar coordinates it collapses:

(\int_{- \infty}^{\infty} e^{- x^{2}} d x)^{2} = \int_{0}^{2 π} \int_{0}^{\infty} e^{- r^{2}} r d r d θ = π \Rightarrow \int_{- \infty}^{\infty} e^{- x^{2}} d x = π .

4 · Deriving t and F from χ² — symmetry does the work

Start from $χ_{k}^{2} = \sum_{i = 1}^{k} Z_{i}^{2}$ , a sum of squared standard normals. The joint density of i.i.d. normals is $\propto e^{- ∥ z ∥^{2} /2}$ — it depends only on the length $∥ z ∥$ , so it is invariant under any rotation. That single symmetry is the engine.

Rotate to an orthonormal basis whose first axis is the all-ones direction $1 / n$ . The first new coordinate carries the sample mean; the remaining $n - 1$ carry the sample variance — and because rotation preserves independence of i.i.d. normals, $\overset{ˉ}{X} ⊥ s^{2}$ with $(n - 1) s^{2} / σ^{2} \sim χ_{n - 1}^{2}$ (Cochran's theorem). From there the two workhorse distributions are just ratios:

T = \frac{Z}{V / k} \sim t_{k}, F = \frac{V _{1} / k _{1}}{V _{2} / k _{2}} \sim F_{k_{1}, k_{2}} .

The forge below builds exactly these by hand — draw the normals, square and sum them into a $χ^{2}$ , form the ratio — and overlays the density the change-of-variables proof predicts. Push the degrees of freedom and watch $t$ approach the normal as $k \to \infty$ .

Sampling Z ~ N(0,1) and V ~ χ²(10), then T = Z / √(V/10). 30,000 draws (bars) against the exact t(10) density (line).

5 · Fourier series — the canonical change of basis

The functions ${e^{in x}}$ form an orthonormal basis of $L^{2} [- π, π]$ , so any square-integrable function is a sum of projections onto them:

f (x) = n = - \infty \sum \infty c_{n} e^{in x}, c_{n} = \frac{1}{2 π} \int_{- π}^{π} f (x) e^{- in x} d x = ⟨ f, e^{in x} ⟩ .

This is the same inner-product geometry as the L² deep dive, now with a basis chosen to diagonalize shifts. Parseval's identity says the change of basis preserves energy: $\frac{1}{2 π} \int ∣ f ∣^{2} = \sum ∣ c_{n} ∣^{2}$ Build a wave from its harmonics below; near a jump the partial sums overshoot by a fixed ~9% no matter how many terms you add — the Gibbs phenomenon, the price of a global basis describing a local discontinuity.

terms: 5

Square wave (grey) rebuilt from 5 odd harmonics (gold). The overshoot at each jump settles in height but never disappears — the Gibbs phenomenon.

6 · How do we know to leap?

The choice of frame isn't magic — it's a short list of tells:

Find the symmetry, align to it. Rotational symmetry → polar or an orthogonal basis (the Gaussian integral, $\overset{ˉ}{X} ⊥ s^{2}$ ). Shift symmetry → Fourier.
A linear operator repeats → diagonalize. Powers, limits, and stationary states are trivial in the eigenbasis.
You know the target's fingerprint → match it. Recognize a distribution by its MGF or characteristic function (the next deep dive).
A hard operation has an easy twin. Convolution → product (Fourier, generating functions); multiplicative → additive (log).
The scale is fighting you → normalize. Standardize, whiten, or variance-stabilize first.

The meta-move is one sentence: choose the representation that turns your question into one you can already answer — and stay honest that the choice was yours, not the world's. That last clause is the QBist reading of a basis, and it's where this thread rejoins the perimeter of pillars.

Study these — derivation cards

Reconstruct before you flip. The matching spaced-repetition deck lives in the Obsidian vault.

Prompt 1 · tap to reveal

State the change-of-variables law for a density, both 1-D and multivariate.

1 / 10