Linear Algebra in a Tiny Neural Network
Trace every shape, bias, activation, and rank change through a two-layer network with a worked NumPy example and an interactive explorer.
Category
Linear Algebra
30articlesin this category.
How Affine Spaces Explain Bias and Decision Boundaries
See how a translation moves a line without changing its direction, and why a neural layer bias can move a classifier boundary away from the origin.
Kernel and Image Show What a Linear Layer Loses and Can Reach
Use a matrix’s kernel and image to identify discarded input directions, reachable outputs, and rank limits in a learned projection.
Matrices Represent Linear Mappings
See why transformed basis vectors become matrix columns, reconstruct a mapping from basis responses, and avoid row-versus-column mistakes in model weights.
Linear Mappings Preserve the Combinations AI Models Need
Test whether a transformation preserves sums and scaling, name its input and output spaces, and separate a neural layer from its activation.
Matrix Rank Counts Independent Information in AI Features
Calculate matrix rank from pivots or features, connect row and column views, and use singular values to spot redundant data features.
Dimension Counts Independent Directions
Separate the number of embedding coordinates from the number of samples, and learn why a matrix shape alone cannot tell you how many independent directions its vectors span.
A Basis Gives AI Vectors a Reversible Coordinate System
Test whether vectors form a basis, calculate coordinates in a new frame, and see how a linear model keeps the same score after a coordinated change.
Span: The Predictions a Linear Model Can Reach
Use vector span to identify every prediction a linear model can produce from its feature columns, then detect a target the feature set cannot express.
Linear Independence Removes Redundancy from AI Features
Use a coefficient equation to spot duplicated feature directions, test linear independence by hand, and check a feature matrix with NumPy.
Subspaces for AI: Test Restricted Feature Families
Learn the subspace test, prove that a plane is closed under vector operations, and use NumPy to inspect lower-dimensional representation directions.
Vector Spaces: Test Whether AI Representations Stay Legal
Test zero, addition, and scalar closure, then use normalized embeddings as a counterexample that separates a useful representation set from a vector space.
Groups, Rings, and Fields: Why AI Operations Need Rules
See how closure, inverses, and distributivity decide which tensor operations are legal, then test the ideas with NumPy.
Numerical Solving for AI: Use Solve Without Explicit Inverses
Solve AI parameter-recovery systems with NumPy, then use residuals, condition numbers, and singularity checks to judge whether the answer is trustworthy.
Row-Echelon Form Reveals Redundant AI Features
Use pivots, free variables, and row-echelon form to diagnose redundant features and non-unique model parameters in a small design matrix.
Gaussian Elimination Preserves the Solution Set
Recover model parameters from a zero-pivot system, then implement safe row operations and residual checks in NumPy.
Identity and Inverse Transformations: Undoing a Feature Map
Use identity matrices and a small inverse to reverse a 2D feature transform, detect unsafe cases, and understand why solve beats explicit inversion.
Transpose for AI: Reorienting Token and Feature Relationships
Transpose a matrix, predict its shape, and switch safely between token-by-feature and feature-by-token views in AI data.
Matrix-Matrix Products for AI: Compose Transformations and Batches
Check matrix shapes, compute a small matrix product, and read the same contraction as a composed map or a batched dense-layer calculation.
Matrix-Vector Products for AI: Two Views of One Dense-Layer Calculation
Compute a matrix times a vector by rows and by columns, implement both views in NumPy, and debug dense-layer shape and operator mistakes.
Matrices for AI: Read Rows, Columns, and Neural Network Shapes
Learn to read matrix dimensions and entries, map rows and columns to dense-layer roles, and catch shape or indexing mistakes in NumPy.
Linear Combinations for AI: How Weighted Vectors Build a New Representation
Scale and combine vectors, inspect every coordinate in NumPy, and see why attention outputs are a special kind of weighted representation.
Scalar Multiplication for AI: Turn a Vector Up, Down, or Around
Scale vectors coordinate by coordinate, predict direction and magnitude, and connect the same operation to embeddings, learning rates, and NumPy.
Vector Addition for AI: How Residual Updates Combine Signals
Learn vector addition component by component, explore its parallelogram geometry, and catch shape, broadcasting, and feature-alignment mistakes in NumPy.
Vectors as AI Data: How Features Turn a Document Into Numbers
Learn to read row and column vectors, build a document feature vector by hand and in NumPy, and catch shape or feature-order mistakes.
No solution, one solution, or many solutions
Learn to classify two-variable linear systems, connect each solution count to parameter identifiability, and handle contradictory or repeated data.
Two Observations, One Parameter Pair: Systems of Equations for AI
See a system as simultaneous constraints, classify two-line intersections, and recover an affine model’s weight and bias from two observations.
Linear Equations for AI: Turn One Formula Into a Geometric Line
Learn how one two-variable linear equation becomes a line, test its points, handle vertical cases, and read a linear classifier score contour.
AlphaEvolve Lowered the Matrix Multiplication Exponent—Not Your GPU Bill
AlphaEvolve helped lower the best matrix multiplication exponent bound. Learn what changed, how it was certified, and why it will not speed up today’s GPUs.
From vectors to embeddings: the geometry behind similarity
Build an intuitive understanding of vectors, dot products, cosine similarity, and why embeddings turn meaning into geometry.