ROC-AUC at 0.97, Precision at 9%: The Base-Rate Math
Work through a 1%-positive detector, turn thresholds into ROC and precision-recall points, and choose an operating point for a 30-alert review budget.
One connected lesson each day
Math for AI
Follow a 180-day path from mathematical notation to regression, PCA, mixture models, and support vector machines. Every lesson shows where the mathematics appears in AI.
Course progress
8 / 180lessons published
Next: Lesson 9, Equations, constraints, and solution sets.
Lessons 1–10
Mathematical Foundations
Published · Lesson 1
Variables, values, and AI models
Read a symbol as a named quantity and separate a variable from its current value.
Published · Lesson 2
Functions as reusable mathematical machines
Evaluate a function and distinguish its input, rule, and output.
Published · Lesson 3
Domain, codomain, and valid model inputs
State where a function accepts inputs and what kind of output it promises.
Published · Lesson 4
Sets, membership, and datasets
Read set-builder and membership notation and use sets to describe data.
Published · Lesson 5
Indices, subscripts, and tensor addresses
Interpret indexed samples and coordinates without confusing identity and value.
Published · Lesson 6
Sums, products, and loss aggregation
Expand sigma and product notation and translate it into loops.
Published · Lesson 7
Powers, roots, and model scaling
Manipulate exponents and roots and recognize scale changes.
Published · Lesson 8
Logarithms turn products into sums
Use logarithms as inverse powers and convert products to sums.
Upcoming · Lesson 9
Equations, constraints, and solution sets
Treat an equation as a constraint and describe all values satisfying it.
Upcoming · Lesson 10
Reading a derivation line by line
Justify each algebraic transformation and identify an unstated assumption.
Lessons 11–38
Linear Algebra
Upcoming · Lesson 11
One linear equation as a geometric line
Convert a two-variable linear equation into a line and test solutions.
Upcoming · Lesson 12
Systems of equations as intersections
Interpret a system as simultaneous constraints and classify its intersections.
Upcoming · Lesson 13
No solution, one solution, or many solutions
Recognize inconsistent, unique, and underdetermined systems.
Upcoming · Lesson 14
Vectors as representations of AI data
Read row and column vectors and interpret coordinates as features.
Upcoming · Lesson 15
Vector addition combines signals
Add vectors componentwise and interpret the resulting displacement or feature mix.
Upcoming · Lesson 16
Scalar multiplication controls strength
Scale a vector and predict changes to direction and magnitude.
Upcoming · Lesson 17
Linear combinations build new vectors
Construct and interpret a weighted sum of vectors.
Upcoming · Lesson 18
Matrices organize linear rules
Read matrix dimensions and interpret rows and columns.
Upcoming · Lesson 19
Matrix-vector products as weighted sums
Compute a matrix-vector product by rows and by columns.
Upcoming · Lesson 20
Matrix-matrix products compose transformations and batches
Check compatible shapes and compute a small matrix product.
Upcoming · Lesson 21
Transpose reorients relationships
Transpose a matrix and explain what swaps semantically.
Upcoming · Lesson 22
Identity and inverse transformations
Recognize identity behavior and verify a small matrix inverse.
Upcoming · Lesson 23
Gaussian elimination preserves the solution set
Apply elementary row operations without changing solutions.
Upcoming · Lesson 24
Row-echelon form reveals structure
Find pivots, free variables, and a general solution.
Upcoming · Lesson 25
Numerical solving without explicit inverses
Use `numpy.linalg.solve` and explain why explicit inversion is usually avoided.
Upcoming · Lesson 26
Why algebraic structures determine valid operations
Distinguish closure, inverse, and distributive assumptions at an intuitive level.
Upcoming · Lesson 27
Vector spaces define legal representations
Test whether a set satisfies vector-space closure rules.
Upcoming · Lesson 28
Subspaces capture restricted feature families
Test subspace conditions and interpret a lower-dimensional region.
Upcoming · Lesson 29
Linear independence removes redundancy
Determine whether vectors contain redundant directions.
Upcoming · Lesson 30
Span describes what a representation can express
Identify the set of vectors reachable by linear combinations.
Upcoming · Lesson 31
Bases give compact coordinate systems
Verify a basis and express a vector in basis coordinates.
Upcoming · Lesson 32
Dimension counts independent directions
Relate dimension to the size of a basis rather than display shape alone.
Upcoming · Lesson 33
Rank measures effective linear information
Compute rank and connect row and column perspectives.
Upcoming · Lesson 34
Linear mappings preserve combinations
Test linearity and describe input and output spaces.
Upcoming · Lesson 35
Matrices represent linear mappings
Derive a mapping matrix from transformed basis vectors.
Upcoming · Lesson 36
Kernel and image explain lost and reachable information
Find simple null-space and image examples and interpret them.
Upcoming · Lesson 37
Affine spaces add offsets to linear structure
Distinguish linear and affine sets and transformations.
Upcoming · Lesson 38
Linear algebra in a tiny neural network
Trace shapes, mappings, bias, rank, and expressivity through a two-layer network.
Lessons 39–54
Analytic Geometry
Upcoming · Lesson 39
Norms measure vector size
Compute common vector norms and check the norm properties.
Upcoming · Lesson 40
L1, L2, and L-infinity see size differently
Explain how different norms change geometry and sensitivity.
Upcoming · Lesson 41
Inner products measure alignment
Compute an inner product and connect sign and magnitude to alignment.
Upcoming · Lesson 42
Dot products turn features into scores
Interpret a dot product as a weighted feature score.
Upcoming · Lesson 43
Distance depends on the chosen geometry
Derive distance from a norm and compare nearest neighbors.
Upcoming · Lesson 44
Angles and cosine similarity
Compute an angle and cosine similarity and handle zero-vector limits.
Upcoming · Lesson 45
Orthogonality means no linear overlap
Test orthogonality and interpret zero inner product geometrically.
Upcoming · Lesson 46
Orthonormal bases simplify coordinates
Verify orthonormality and compute coordinates with inner products.
Upcoming · Lesson 47
Gram–Schmidt builds orthogonal directions
Execute Gram–Schmidt on two or three vectors and identify instability.
Upcoming · Lesson 48
Orthogonal complements capture what is left
Describe the directions orthogonal to a subspace.
Upcoming · Lesson 49
Functions can have inner products too
Interpret function similarity through an integral or discrete sum.
Upcoming · Lesson 50
Projection finds the nearest point
Compute projection onto a line and explain the perpendicular residual.
Upcoming · Lesson 51
Projection matrices transform whole datasets
Construct and check a simple projection matrix.
Upcoming · Lesson 52
Least squares is a projection problem
Derive normal-equation geometry without relying on explicit inversion.
Upcoming · Lesson 53
Rotations preserve length and angle
Build a 2D rotation matrix and verify its invariants.
Upcoming · Lesson 54
Geometry of similarity search
Choose a metric, normalization, and projection strategy for a retrieval problem.
Lessons 55–72
Matrix Decompositions
Upcoming · Lesson 55
Determinants measure oriented volume
Compute 2D determinants and interpret sign and scale.
Upcoming · Lesson 56
Singular matrices collapse information
Connect zero determinant, rank loss, and non-invertibility.
Upcoming · Lesson 57
Trace summarizes diagonal action
Compute trace and use its linear and cyclic properties safely.
Upcoming · Lesson 58
Eigenvectors keep their direction
Verify an eigenpair and distinguish direction from scale.
Upcoming · Lesson 59
Eigenvalues measure directional growth
Interpret eigenvalue sign and magnitude, including zero and repeated cases.
Upcoming · Lesson 60
Characteristic polynomials find eigenvalues
Solve a 2×2 characteristic equation and check the result.
Upcoming · Lesson 61
Eigenvectors in Markov and ranking systems
Explain how repeated linear updates approach a stable direction.
Upcoming · Lesson 62
Cholesky factorization exploits positive definiteness
Factor a small positive-definite matrix and recognize eligibility conditions.
Upcoming · Lesson 63
Diagonalization decouples a transformation
Express a diagonalizable matrix in its eigenbasis.
Upcoming · Lesson 64
Symmetric matrices have friendly eigenstructure
State and use the real, orthogonal eigenstructure of symmetric matrices.
Upcoming · Lesson 65
Singular value decomposition factorizes any matrix
Interpret SVD as rotate, scale, rotate and track shapes.
Upcoming · Lesson 66
Singular values rank directions by strength
Connect singular values to rank, conditioning, and captured signal.
Upcoming · Lesson 67
Pseudoinverses solve nonsquare systems
Use the SVD pseudoinverse and explain discarded zero directions.
Upcoming · Lesson 68
Low-rank approximation keeps the strongest patterns
Construct a truncated SVD and quantify reconstruction error.
Upcoming · Lesson 69
Matrix compression for model weights
Calculate parameter savings and identify quality tradeoffs.
Upcoming · Lesson 70
Conditioning predicts numerical sensitivity
Compute a condition number and predict error amplification.
Upcoming · Lesson 71
NumPy factorizations and reconstruction checks
Choose and verify Cholesky, eigendecomposition, or SVD in code.
Upcoming · Lesson 72
How decompositions power modern AI systems
Select a decomposition for ranking, covariance, solving, or compression.
Lessons 73–98
Vector Calculus
Upcoming · Lesson 73
Change, slopes, and local predictions
Interpret slope as local change rather than only rise over run.
Upcoming · Lesson 74
Limits make instantaneous change precise
Explain the derivative limit and distinguish smaller steps from zero steps.
Upcoming · Lesson 75
Derivative rules avoid repeating limits
Apply sum, product, quotient, and power rules with assumptions.
Upcoming · Lesson 76
The chain rule traces composed change
Differentiate a composition and track intermediate sensitivities.
Upcoming · Lesson 77
Derivatives of common activation functions
Derive and compare sigmoid, tanh, and ReLU derivatives, including nondifferentiable points.
Upcoming · Lesson 78
Partial derivatives hold other inputs fixed
Compute partial derivatives and state what remains fixed.
Upcoming · Lesson 79
Gradients collect steepest local change
Build a gradient vector and interpret its direction and units.
Upcoming · Lesson 80
Directional derivatives query any direction
Compute change along a chosen direction from a gradient.
Upcoming · Lesson 81
Jacobians map vector-to-vector sensitivity
Construct a Jacobian and track its input/output shape.
Upcoming · Lesson 82
The Jacobian chain rule composes layers
Multiply Jacobians in the correct order and check shapes.
Upcoming · Lesson 83
Matrix gradients track every parameter
Interpret derivatives with respect to a matrix and verify dimensions.
Upcoming · Lesson 84
Trace tricks simplify matrix derivatives
Rewrite scalar matrix objectives with trace identities and differentiate them.
Upcoming · Lesson 85
Computation graphs make dependencies visible
Draw dependencies and assign local derivatives to graph edges.
Upcoming · Lesson 86
Forward-mode automatic differentiation propagates tangents
Execute forward accumulation and explain when it is efficient.
Upcoming · Lesson 87
Reverse-mode automatic differentiation propagates adjoints
Execute a reverse pass and explain why scalar losses favor it.
Upcoming · Lesson 88
Backpropagation through a dense layer
Derive input, weight, and bias gradients with correct batch shapes.
Upcoming · Lesson 89
Gradient checking catches derivative bugs
Compare analytic and finite-difference gradients with scale-aware error.
Upcoming · Lesson 90
PyTorch autograd records the graph
Use tensors, `requires_grad`, and backward while explaining the graph.
Upcoming · Lesson 91
Hessians measure multivariate curvature
Construct a small Hessian and interpret curvature directions.
Upcoming · Lesson 92
Second derivatives classify critical points
Use Hessian eigenvalues to classify local minima, maxima, and saddles.
Upcoming · Lesson 93
Linearization approximates a model locally
Build a first-order approximation and quantify local error.
Upcoming · Lesson 94
Taylor series add curvature information
Construct first- and second-order Taylor approximations.
Upcoming · Lesson 95
Vanishing gradients from repeated chain rules
Explain how multiplying small derivatives suppresses learning signals.
Upcoming · Lesson 96
Exploding gradients and stability controls
Explain exploding products and evaluate clipping and initialization responses.
Upcoming · Lesson 97
Attention gradients connect token influence
Trace how one token can affect another through differentiable attention weights.
Upcoming · Lesson 98
The calculus of a tiny training step
Connect forward evaluation, loss, gradients, curvature, and parameter updates.
Lessons 99–126
Probability & Distributions
Upcoming · Lesson 99
Uncertainty needs a mathematical language
Distinguish uncertainty about outcomes from lack of numerical precision.
Upcoming · Lesson 100
Sample spaces, events, and outcomes
Define outcomes and events and combine events with set operations.
Upcoming · Lesson 101
Probability axioms constrain valid assignments
Apply nonnegativity, normalization, and additivity to detect invalid probabilities.
Upcoming · Lesson 102
Conditional probability updates the reference set
Compute conditional probability and explain the changed denominator.
Upcoming · Lesson 103
Random variables map outcomes to numbers
Treat a random variable as a function and state its possible values.
Upcoming · Lesson 104
Discrete distributions assign probability mass
Build and normalize a probability mass function.
Upcoming · Lesson 105
Continuous distributions use density, not point mass
Interpret probability density and areas over intervals.
Upcoming · Lesson 106
Cumulative distributions answer threshold questions
Read and construct a cumulative distribution function.
Upcoming · Lesson 107
Sum and product rules build and reduce joint probabilities
Expand joint probabilities and marginalize hidden variables.
Upcoming · Lesson 108
Bayes' theorem reverses a condition
Derive Bayes' theorem and identify prior, likelihood, evidence, and posterior.
Upcoming · Lesson 109
Base rates change what a model alert means
Explain why sensitivity alone does not determine posterior probability.
Upcoming · Lesson 110
Expectation is a probability-weighted average
Compute expectations for discrete variables and interpret the continuous analogue.
Upcoming · Lesson 111
Variance measures spread around expectation
Compute variance and standard deviation and distinguish scale from bias.
Upcoming · Lesson 112
Covariance tracks joint movement
Compute covariance and interpret sign without claiming causation.
Upcoming · Lesson 113
Covariance matrices summarize many features
Build a covariance matrix and explain symmetry and diagonal entries.
Upcoming · Lesson 114
Independence is stronger than zero covariance
Test independence and produce a zero-covariance dependent counterexample.
Upcoming · Lesson 115
Empirical estimates learn statistics from samples
Estimate mean and variance and distinguish population from sample quantities.
Upcoming · Lesson 116
Gaussian distributions link location and scale
Read the Gaussian density and predict effects of mean and variance.
Upcoming · Lesson 117
Standardization creates comparable coordinates
Standardize values and interpret z-scores without assuming normality.
Upcoming · Lesson 118
Multivariate Gaussians shape uncertainty with covariance
Interpret mean vectors, covariance matrices, and density ellipses.
Upcoming · Lesson 119
Gaussian conditionals and marginals
Extract marginal and conditional Gaussian behavior.
Upcoming · Lesson 120
Sampling correlated Gaussians with Cholesky
Transform standard noise into samples with a target covariance.
Upcoming · Lesson 121
Exponential families share a common form
Identify natural parameters, sufficient statistics, and normalization conceptually.
Upcoming · Lesson 122
Conjugate priors make Bayesian updates tractable
Explain conjugacy and update a Beta-Bernoulli model.
Upcoming · Lesson 123
Maximum likelihood connects observed data to parameters
Construct and optimize a simple likelihood and log-likelihood.
Upcoming · Lesson 124
Change of variables transforms densities
Transform a density and explain the Jacobian determinant correction.
Upcoming · Lesson 125
Inverse-transform sampling turns uniform noise into data
Generate samples with an inverse CDF and verify their distribution.
Upcoming · Lesson 126
Probability in a classifier pipeline
Connect likelihoods, priors, posteriors, calibration questions, and evaluation uncertainty.
Lessons 127–142
Continuous Optimization
Upcoming · Lesson 127
Objective functions turn learning into search
Define an objective, parameters, and feasible set and distinguish minimization from evaluation.
Upcoming · Lesson 128
Gradient descent follows local downhill information
Derive the negative-gradient update and execute steps by hand.
Upcoming · Lesson 129
Learning rates control step size
Predict convergence, oscillation, and divergence as step size changes.
Upcoming · Lesson 130
Loss landscapes contain valleys, plateaus, and saddles
Identify landscape features and their effect on first-order updates.
Upcoming · Lesson 131
Stochastic gradients trade noise for speed
Compare batch, mini-batch, and stochastic gradient estimates.
Upcoming · Lesson 132
Momentum accumulates a velocity
Execute momentum updates and explain damping in narrow valleys.
Upcoming · Lesson 133
Adaptive optimizers rescale coordinate updates
Explain moving moments and coordinate-wise scaling without treating Adam as magic.
Upcoming · Lesson 134
Constrained optimization separates goals and rules
State equality constraints and distinguish objective contours from feasible points.
Upcoming · Lesson 135
Lagrange multipliers balance objective and constraint
Derive stationarity for one equality constraint and interpret gradient alignment.
Upcoming · Lesson 136
Regularization can act like a soft constraint
Relate penalties to constrained solutions while stating the equivalence conditions.
Upcoming · Lesson 137
Convex sets contain the lines between points
Test convexity of simple sets using line segments.
Upcoming · Lesson 138
Convex functions have no bad local minima
Apply geometric and Hessian tests for convexity.
Upcoming · Lesson 139
Convexity gives optimization certificates
Explain why first-order conditions can certify a global solution.
Upcoming · Lesson 140
Optimization in PyTorch without a black box
Write an explicit training loop and inspect gradients and updates.
Upcoming · Lesson 141
Debugging training with mathematical signals
Diagnose loss, gradient norm, update ratio, and curvature clues without overclaiming.
Upcoming · Lesson 142
Design an optimizer for a toy model
Select an update rule, learning rate, constraint or penalty, and stopping evidence.
Lessons 143–151
Models & Data
Upcoming · Lesson 143
Data, models, and learning rules play different roles
Separate observations, hypotheses, parameters, predictions, and fitting rules.
Upcoming · Lesson 144
Supervised learning approximates an unknown map
Express supervised data and a prediction function with correct indexing.
Upcoming · Lesson 145
Empirical risk turns examples into an objective
Derive empirical risk from per-example loss and distinguish it from population risk.
Upcoming · Lesson 146
Loss functions encode which errors matter
Compare squared, absolute, and classification losses by consequence and gradient.
Upcoming · Lesson 147
Parameter estimation connects probability and optimization
Frame maximum likelihood and MAP as optimization problems.
Upcoming · Lesson 148
Probabilistic modeling specifies a generative story
Factor a simple joint distribution and distinguish parameters from latent variables.
Upcoming · Lesson 149
Inference asks about hidden quantities
State an inference query and identify what must be conditioned or marginalized.
Upcoming · Lesson 150
Directed graphs encode conditional structure
Read a directed graphical model and write its joint factorization.
Upcoming · Lesson 151
Model selection balances fit and complexity
Distinguish training fit, validation choice, and probabilistic evidence.
Lessons 152–159
Linear Regression
Upcoming · Lesson 152
Linear regression maps features to numeric targets
Write scalar and matrix forms of linear regression and track dimensions.
Upcoming · Lesson 153
Least-squares estimation from projection geometry
Derive the least-squares estimator and interpret residual orthogonality.
Upcoming · Lesson 154
Maximum likelihood explains squared error under Gaussian noise
Derive squared-error regression from a Gaussian likelihood and state assumptions.
Upcoming · Lesson 155
Regularized regression controls unstable weights
Derive ridge regression and connect the penalty to conditioning and bias.
Upcoming · Lesson 156
Bayesian linear regression puts a distribution on weights
Update a Gaussian prior into a parameter posterior.
Upcoming · Lesson 157
Posterior predictive uncertainty varies by input
Separate observation noise from parameter uncertainty in predictions.
Upcoming · Lesson 158
Linear regression implementation and diagnostics
Implement stable fitting and inspect residuals, conditioning, and uncertainty.
Upcoming · Lesson 159
When a linear model is the right baseline
Decide when interpretability and data limits favor linear regression and identify failure modes.
Lessons 160–167
Principal Component Analysis
Upcoming · Lesson 160
Dimensionality reduction preserves selected structure
State what reduction keeps, what it discards, and why the criterion matters.
Upcoming · Lesson 161
PCA finds directions of maximum variance
Derive the first principal direction from covariance eigenvectors.
Upcoming · Lesson 162
PCA is also a minimum-error projection
Connect maximum variance and minimum reconstruction error.
Upcoming · Lesson 163
SVD computes PCA without forming covariance
Relate centered-data SVD to covariance eigenvectors and singular values.
Upcoming · Lesson 164
PCA in high-dimensional AI data
Choose a computation path when features outnumber samples and interpret effective rank.
Upcoming · Lesson 165
Practical PCA: centering, scaling, and component choice
Build a leakage-safe PCA pipeline and choose components using evidence.
Upcoming · Lesson 166
Probabilistic PCA introduces latent variables
Interpret PCA as a latent-variable generative model with noise.
Upcoming · Lesson 167
PCA for embedding inspection, with caveats
Use PCA for inspection without treating a 2D plot as complete evidence.
Lessons 168–173
Gaussian Mixture Models
Upcoming · Lesson 168
Mixtures model data with multiple overlapping groups
Write a mixture density and distinguish components from observed groups.
Upcoming · Lesson 169
Latent assignments turn mixtures into a hidden-variable model
Introduce latent component indicators and compute responsibilities conceptually.
Upcoming · Lesson 170
Mixture likelihood couples unknown parameters
Construct the GMM likelihood and explain why direct maximization is difficult.
Upcoming · Lesson 171
Expectation–maximization alternates inference and fitting
Execute E and M steps and identify what each holds fixed.
Upcoming · Lesson 172
Implement a GMM and watch EM converge
Implement stable log responsibilities and monitor likelihood.
Upcoming · Lesson 173
GMM failure modes and model selection
Diagnose initialization sensitivity, singular covariance, and wrong component count.
Lessons 174–180
Support Vector Machines
Upcoming · Lesson 174
Hyperplanes turn dot products into decision boundaries
Write a separating hyperplane and interpret its normal vector and bias.
Upcoming · Lesson 175
Margin measures classification robustness
Derive point-to-hyperplane distance and identify support vectors.
Upcoming · Lesson 176
The primal SVM balances margin and classification errors
Formulate hard- and soft-margin objectives and interpret slack variables.
Upcoming · Lesson 177
The dual SVM centers training examples
Follow the Lagrangian route to a dual representation and read support-vector coefficients.
Upcoming · Lesson 178
Kernels create nonlinear boundaries through inner products
Explain the kernel trick and test basic kernel intuition without claiming an explicit feature map is always known.
Upcoming · Lesson 179
Train and diagnose an SVM
Train linear and kernel SVMs and diagnose scaling, penalty, and kernel-width effects.
Upcoming · Lesson 180
From mathematical foundations to ML models
Trace how algebra, geometry, calculus, probability, and optimization jointly determine model behavior.
Beyond the sequence
Supplementary Math for AI
Temperature Scaling and ECE: Calibrate Model Confidence Step by Step
Work through softmax, fit a temperature on six predictions, compute ECE, and see why a better likelihood can still produce a worse binned calibration score.
Cross-Model KV Cache Transfer Is Fast—and Surprisingly Pair-Specific
NVIDIA researchers report a 25× component speedup for one KV-cache mapping, while several other model pairs lost much of the target model’s task accuracy.
AlphaEvolve Lowered the Matrix Multiplication Exponent—Not Your GPU Bill
AlphaEvolve helped lower the best matrix multiplication exponent bound. Learn what changed, how it was certified, and why it will not speed up today’s GPUs.
From vectors to embeddings: the geometry behind similarity
Build an intuitive understanding of vectors, dot products, cosine similarity, and why embeddings turn meaning into geometry.