Backporch
Writing · a learning atlas
← All writing

Modern Multivariate Statistical Techniques

A learning atlas · Alan Julian Izenman

The whole multivariate toolkit as one map. Izenman's 17 chapters, regrouped from the book's order into the strata a practitioner works in — foundations, regression, classification, ensembles, dimension reduction, latent structure — each lit to where the idea already lives here: a live demo, a research explore, a pillar, or a lab. A chapter at a time; the map fills in as I read.

6 of 17 chapters already live somewhere on the site · 0 written up · the map fills in as I read.

Foundations

the objects everything rides on — data, random vectors, densities
  1. 1Introduction and Previewthe multivariate toolkit as one map — regression, classification, manifold learningplanned
  2. 2Data and Databasestidy data, silos, and the data-wrangling bench on /proofsplanned
  3. 3Random Vectors and Matricesdeep divecovariance as geometry → the L² deep dive; spectra via the random-matrix labdeep dive →
  4. 4Nonparametric Density Estimationlabkernels & bandwidth → attention as kernel regression on /architectureslab →

Regression

predict a number; then assess and select the model that does it
  1. 5Model Assessment & Selection in Multiple Regressionbias–variance, cross-validation, AIC/BIC → the scaling & regularization pillarplanned
  2. 6Multivariate Regressionreduced-rank & ridge; the response as a vectorplanned

Classification

draw the boundary — discriminants, trees, nets, and the widest margin
  1. 8Linear Discriminant AnalysisLDA as a generative classifier; Fisher's ratio (the F-test geometry on /proofs)planned
  2. 9Recursive Partitioning & Tree-Based MethodsCART → the committee machines of Ch 14planned
  3. 10Artificial Neural Networkslabbackprop & network design → the whole /architectures pagelab →
  4. 11Support Vector Machinesdemothe max-margin classifier & the kernel trick — live & viewer-gated: boundary, margin, support vectors, scorecardclient demo →

Ensembles

many weak learners voting into one strong committee
  1. 14Committee Machinesbagging, boosting, random forests — the ensembles behind most tabular MLplanned

Dimension reduction & manifolds

fewer coordinates that keep the shape — linear through nonlinear
  1. 7Linear Dimensionality Reductiondeep divePCA / SVD = eigen → the Eigenbook and L² deep divedeep dive →
  2. 13Multidimensional Scaling & Distance Geometrydistances → coordinates; the classical MDS inside Isomap, and the bridge to TDAplanned
  3. 16Nonlinear Dimensionality Reduction & Manifold Learningexplorekernel PCA, IsoMap, LLE, Laplacian & Hessian eigenmaps — live: PCA vs Isomap vs Laplacian on a swiss rollexplore →
  4. 17Correspondence Analysisχ² geometry of contingency tables — PCA's categorical cousinplanned

Clustering & latent structure

structure with no labels — the groups, and the hidden factors behind them
  1. 12Cluster Analysisk-means, hierarchical, mixtures → the manifold clusters on /computeplanned
  2. 15Latent Variable Models for Blind Source SeparationICA & factor models — unmixing signals; strong client use casesplanned