Matrix-Matrix Products for AI: Compose Transformations and Batches

A dense layer does the same kind of calculation for one input or for many inputs at once. For one input, a matrix-vector product combines a row of weights with a feature vector. For a batch, the rows of an input matrix are combined with every output rule. Matrix-matrix multiplication is the compact notation that makes this reuse possible.

By the end of this lesson, you will be able to check compatible shapes, compute a small matrix product, explain its output dimensions, and interpret the same product as either composed transformations or a batch of matrix-vector products.

Learning outcome: Check compatible shapes, compute a small matrix product, explain output dimensions, and interpret the same product as composed transformations or a batch of matrix-vector products.

Lesson 19: Matrix-vector products as weighted sums treated one input at a time. The official Mathematics for Machine Learning companion develops matrix products in Section 2.2 of the book. The examples, code, visual, and assessments here are original applications of that definition.

The inner dimensions decide whether a product exists

Let

$$A\in\mathbb{R}^{m\times n}, \qquad B\in\mathbb{R}^{n\times p}.$$

The product $C=AB$ is defined because the number of columns in $A$ equals the number of rows in $B$. Its shape keeps the two outside dimensions:

$$\underbrace{A}_{m\times n} \underbrace{B}_{n\times p} = \underbrace{C=AB}_{m\times p}.$$

The shared dimension $n$ tells us how many aligned pairs are multiplied and added for each output entry. The outer dimensions $m$ and $p$ tell us how many rows and columns the result has. In other words, the output shape is $(m,p)$, not $(m,n)$ and not $(n,p)$.

This rule is a contraction over the shared axis. It is not an instruction to multiply entries in the same positions. That distinction matters in code: NumPy documents numpy.matmul as the matrix product and uses @ as its operator shorthand in the ordinary two-dimensional case (documentation).

A shape-flow simulator keeps the axes visible

The following visual starts with a complete, compatible product. Choose a different preset to see other valid outer shapes, switch between the direct row-by-column reading and the batch reading, or use the native controls to trace one output entry. Every state also has a semantic table below it, so the calculation remains readable without JavaScript, color, or animation.

Explore the shape flow

Compatible shape presetA (2 × 3) × B (3 × 2) → C (2 × 2)P (2 × 2) × Q (2 × 3) → R (2 × 3)R (3 × 1) × S (1 × 2) → T (3 × 2)

Each option has matching inner dimensions; the outside dimensions determine its output shape.

Read the same product as

Row × column productRow-wise batch

Trace one output entryC11 (row 1, column 1)C12 (row 1, column 2)C21 (row 2, column 1)C22 (row 2, column 2)

Shared-dimension terms

3 of 3 products

Move the slider with a pointer or keyboard. The tables remain available as a complete static reference.

Tracing C11: row 1, column 1; 3 of 3 products included.

Left matrix × Right matrix → Output matrix

A2 × 3

1-12201

×

B3 × 2

213-104

=

C2 × 2

-1^★1046

Row-by-column trace

C11 = 1 × 2 + (-1) × 3 + 2 × 0 = -1

Running total: -1 · complete dot product

Semantic table: every output entry, its left row, right column, and final dot product for A × B = C

Output entry	Left row	Right column	Dot product
C11 (row 1, column 1)	[1, -1, 2]	[2, 3, 0]	-1
C12 (row 1, column 2)	[1, -1, 2]	[1, -1, 4]	10
C21 (row 2, column 1)	[2, 0, 1]	[2, 3, 0]	4
C22 (row 2, column 2)	[2, 0, 1]	[1, -1, 4]	6

Semantic table: the same multiplication read as a row-wise batch for Bᵀ × Aᵀ = Cᵀ

Batch row	Input row (a right-matrix column)	Output row
1	[2, 3, 0]	[-1, 4]
2	[1, -1, 4]	[10, 6]

The tables are rendered with the lesson and stay visible without JavaScript. Focused controls, text labels, and the highlighted star provide non-color cues for the interactive state.

The server-rendered first state shows the full product before the optional client-side interaction becomes visible. The slider reveals partial sums, not new mathematics: it only decides how many terms of the same dot product are currently displayed. In batch mode, the rows shown are complete examples, and the table is the permanent fallback for all of them.

A small product reveals four dot products

Consider the original matrices

$$A= \begin{bmatrix} 1 & -1 & 2\\ 2 & 0 & 1 \end{bmatrix}, \qquad B= \begin{bmatrix} 2 & 1\\ 3 & -1\\ 0 & 4 \end{bmatrix}.$$

The shape check comes first:

$$(2\times3)(3\times2)\longrightarrow(2\times2).$$

The two inner dimensions are both $3$, so the product exists. The two outer dimensions predict four output entries, each formed from three scalar multiplications.

The location of each answer is part of the result. For example, $c_{12}$ is not “the second multiplication” in a flattened list; it is the entry at row $1$, column $2$, and it always pairs row $1$ of $A$ with column $2$ of $B$.

The same product composes transformations in a fixed order

The matrix shapes also describe domains and codomains. Here $B$ maps $\mathbb{R}^{2}$ to $\mathbb{R}^{3}$, while $A$ maps $\mathbb{R}^{3}$ to $\mathbb{R}^{2}$:

$$\mathbb{R}^{2} \xrightarrow{\;B\;} \mathbb{R}^{3} \xrightarrow{\;A\;} \mathbb{R}^{2}.$$

Therefore $AB$ is the single matrix for “apply $B$, then apply $A$.” For

$$\boldsymbol{x}=\begin{bmatrix}1\\2\end{bmatrix},$$

the intermediate vector is

$$B\boldsymbol{x} = \begin{bmatrix} 2\cdot1+1\cdot2\\ 3\cdot1+(-1)\cdot2\\ 0\cdot1+4\cdot2 \end{bmatrix} = \begin{bmatrix}4\\1\\8\end{bmatrix}.$$

Applying $A$ next gives

$$A(B\boldsymbol{x}) = \begin{bmatrix} 1\cdot4+(-1)\cdot1+2\cdot8\\ 2\cdot4+0\cdot1+1\cdot8 \end{bmatrix} = \begin{bmatrix}19\\16\end{bmatrix}.$$

The matrix product gives the same endpoint without materializing the middle vector:

$$C\boldsymbol{x} = \begin{bmatrix}-1&10\\4&6\end{bmatrix} \begin{bmatrix}1\\2\end{bmatrix} = \begin{bmatrix}19\\16\end{bmatrix}.$$

The order is not decorative. The reverse product is valid here, but it has a different shape and meaning:

$$BA= \begin{bmatrix} 4 & -2 & 5\\ 1 & -3 & 5\\ 8 & 0 & 4 \end{bmatrix}\in\mathbb{R}^{3\times3}.$$

$BA$ maps $\mathbb{R}^{3}$ to $\mathbb{R}^{3}$, so it cannot be substituted for $AB$ on the two-coordinate vector $\boldsymbol{x}$. The book’s Section 2.2 also emphasizes that matrix multiplication is generally non-commutative; the domain-and-codomain picture explains why swapping the order changes the calculation (book PDF).

The contraction is a batch of matrix-vector products

Suppose three examples each have three features. Put one example per row:

$$X= \begin{bmatrix} 1&2&0\\ 0&1&2\\ -1&0&1 \end{bmatrix}\in\mathbb{R}^{3\times3}.$$

Use the earlier $A$ as a dense-layer weight matrix and call it $W$. It has shape $(2,3)$: three input features and two output scores. A row-oriented batch calculation needs the transpose $W^{\mathsf T}$:

$$XW^{\mathsf T} = \begin{bmatrix} 1&2&0\\ 0&1&2\\ -1&0&1 \end{bmatrix} \begin{bmatrix} 1&2\\ -1&0\\ 2&1 \end{bmatrix} = \begin{bmatrix} -1&2\\ 3&2\\ 1&-1 \end{bmatrix}.$$

The shape calculation is

$$(3\times3)(3\times2)\longrightarrow(3\times2).$$

The first dimension, $3$, is the number of examples. The second dimension, $2$, is the number of output scores per example. Adding a bias vector

$$\boldsymbol{b}=\begin{bmatrix}1&-1\end{bmatrix}$$

to every row gives

$$Y=XW^{\mathsf T}+\boldsymbol{b} = \begin{bmatrix} 0&1\\ 4&1\\ 2&-2 \end{bmatrix}.$$

NumPy’s broadcasting rules allow a shape $(2,)$ bias to align with the last axis of each $(3,2)$ row (broadcasting documentation). The matrix product itself is still the contraction; the bias addition is a separate elementwise operation.

This row-batch convention is also the convention exposed by torch.nn.Linear: with in_features=3 and out_features=2, PyTorch stores a weight of shape $(2,3)$ and computes $y=xW^{\mathsf T}+b$. An input with shape $(3,3)$ is therefore returned with shape $(3,2)$. The transpose bridges the mathematical column-vector convention and the implementation convention in which each sample is a row. For tensors with more batch dimensions, torch.matmul keeps the final two axes for the matrix product and broadcasts compatible leading batch axes.

There is another way to see the same result. The columns of $B$ are two three-coordinate inputs for the map $A$. Put those inputs into rows by transposing $B$:

$$B^{\mathsf T}A^{\mathsf T} = \begin{bmatrix} 2&3&0\\ 1&-1&4 \end{bmatrix} \begin{bmatrix} 1&2\\ -1&0\\ 2&1 \end{bmatrix} = \begin{bmatrix} -1&4\\ 10&6 \end{bmatrix} =C^{\mathsf T}.$$

Each row of $B^{\mathsf T}$ is one column of $B$, and each row of $C^{\mathsf T}$ is the corresponding output of $A$ on that input. The identity $(AB)^{\mathsf T}=B^{\mathsf T}A^{\mathsf T}$ is therefore the bridge between “columns as separate inputs” and “rows as a batch.”

NumPy makes the contraction testable

The following small program writes the three nested loops explicitly and checks them against NumPy. It also checks the batch identity and keeps the elementwise operator visibly separate. NumPy’s quickstart introduces the same distinction between elementwise operations and matrix products.

import numpy as np

A = np.array([[1, -1, 2], [2, 0, 1]])
B = np.array([[2, 1], [3, -1], [0, 4]])

def matrix_product_by_loops(left, right):
    if left.shape[1] != right.shape[0]:
        raise ValueError("inner dimensions do not match")

    result = np.zeros((left.shape[0], right.shape[1]), dtype=int)
    for i in range(left.shape[0]):
        for j in range(right.shape[1]):
            for ell in range(left.shape[1]):
                result[i, j] += left[i, ell] * right[ell, j]
    return result

C_by_loops = matrix_product_by_loops(A, B)
C = A @ B
np.testing.assert_array_equal(C_by_loops, C)
np.testing.assert_array_equal(B.T @ A.T, C.T)

print(C)
print("elementwise A * B.T has shape", (A * B.T).shape)

try:
    A @ np.ones((2, 2), dtype=int)
except ValueError as error:
    print(type(error).__name__, "for incompatible inner dimensions")

The program prints

[[-1 10]
 [ 4  6]]
elementwise A * B.T has shape (2, 3)
ValueError for incompatible inner dimensions

The invalid call fails because $A$ has three columns while the second operand has two rows. The elementwise expression A * B.T does not fail here because both arrays have shape $(2,3)$; it produces

$$A\mathbin{*}B^{\mathsf T} = \begin{bmatrix}2&-3&0\\2&0&4\end{bmatrix},$$

which has no row-by-column summation. NumPy documents multiply as element-by-element multiplication (reference). The fact that two operators accept the same operands is not evidence that they represent the same mathematical operation.

Identity and zero results are useful boundary cases

Two edge cases sharpen the shape rule. First, a compatible identity matrix changes no values:

$$I_2C=C=CI_2, \qquad I_2=\begin{bmatrix}1&0\\0&1\end{bmatrix}.$$

The identity on the left has to have two rows and columns here because $C$ is $(2,2)$. In general, $I_mA=A$ for $A\in\mathbb{R}^{m\times n}$ and $AI_n=A$ when the dimensions fit. Identity matrices preserve a valid product; they do not repair an invalid one.

Second, a valid product can be all zeros because terms cancel. Let

$$P=\begin{bmatrix}1&-1\\0&0\end{bmatrix}, \qquad Q=\begin{bmatrix}3&3\\3&3\end{bmatrix}.$$

Then

$$PQ=\begin{bmatrix}1\cdot3+(-1)\cdot3&1\cdot3+(-1)\cdot3\\0&0\end{bmatrix} =\begin{bmatrix}0&0\\0&0\end{bmatrix}.$$

The inner dimensions match, so the zero result is a calculation, not a shape error. When debugging a dense layer, check both kinds of evidence: the shape contract tells you whether the operation is legal, while the actual values tell you whether cancellation, a zero weight, or a data issue explains the output.

The reusable habit is short: name the left shape, name the right shape, match the inner dimensions, keep the outer dimensions, then compute each output entry from a left row and a right column. That habit scales from the small worked example to X @ W.T + b, to a composition of learned maps, and to larger batched tensor operations.

Check your understanding

Question 1

If L has shape 4 × 3 and R has shape 3 × 5, what shape does LR have, and how many scalar multiplications are needed for the full product?

Show the step-by-step solution

Step 1: check the shared axis. The left matrix has 3 columns and the right matrix has 3 rows, so the inner dimensions match.

Step 2: keep the outer axes. The product has shape $(4,5)$.

Step 3: count the work. There are $4\cdot5=20$ output entries. Each one uses a length-3 dot product, so the direct calculation uses $20\cdot3=60$ scalar multiplications.

Question 2

Compute P Q for P = [[1, 0, −2], [2, 1, 1]] and Q = [[3, 1], [0, −2], [4, 2]].

Show the step-by-step solution

Step 1: predict the shape. The shapes are $(2,3)$ and $(3,2)$, so the result must be $(2,2)$.

Step 2: compute row 1. Its entries are $1\cdot3+0\cdot0+(-2)\cdot4=-5$ and $1\cdot1+0\cdot(-2)+(-2)\cdot2=-3$.

Step 3: compute row 2. Its entries are $2\cdot3+1\cdot0+1\cdot4=10$ and $2\cdot1+1\cdot(-2)+1\cdot2=2$.

Therefore

$$PQ=\begin{bmatrix}-5&-3\\10&2\end{bmatrix}.$$

Question 3

A dense layer stores W with shape (2, 3), receives X with shape (4, 3), and has bias b with shape (2,). What product and output shape should you use?

Show the step-by-step solution

Step 1: bridge the conventions. Since examples are rows, transpose the weight matrix: $W^{\mathsf T}$ has shape $(3,2)$.

Step 2: multiply the batch. $XW^{\mathsf T}$ has shape $(4,3)(3,2)\to(4,2)$.

Step 3: add the bias row-wise. The bias has two entries, matching the two output features, so $XW^{\mathsf T}+b$ remains $(4,2)$. The leading 4 counts examples; it is not part of the learned weight shape.

Question 4

For the lesson's A and B, why does Bᵀ Aᵀ equal (A B)ᵀ, and what does each row of Bᵀ represent?

Show the step-by-step solution

Step 1: transpose the factors in reverse order. Transposing $AB$ gives $(AB)^{\mathsf T}=B^{\mathsf T}A^{\mathsf T}$.

Step 2: identify the rows. Each row of $B^{\mathsf T}$ is one column of $B$, so it is a three-coordinate input for the map $A$.

Step 3: read the outputs. Each corresponding row of $B^{\mathsf T}A^{\mathsf T}$ is the output for that input under the row-batch convention. Numerically,

$$B^{\mathsf T}A^{\mathsf T} =\begin{bmatrix}-1&4\\10&6\end{bmatrix} =\begin{bmatrix}-1&10\\4&6\end{bmatrix}^{\mathsf T}.$$

Question 5

Using the lesson's A, B, and x = [1, 2]ᵀ, which map acts first in (A B)x, and what is the final vector?

Show the step-by-step solution

Step 1: read the order. The rightmost factor acts first, so compute $B\boldsymbol{x}$ before applying $A$.

Step 2: compute the intermediate vector.

$$B\boldsymbol{x}=\begin{bmatrix}4\\1\\8\end{bmatrix}.$$

Step 3: apply the second map.

$$A(B\boldsymbol{x}) =\begin{bmatrix}19\\16\end{bmatrix}.$$

Thus $AB$ represents “$B$ first, then $A$,” and the final vector is $[19,16]^{\mathsf T}$.

Question 6

Why is A * Bᵀ not the same operation as A B, even though both expressions are accepted for the lesson's arrays?

Show the step-by-step solution

Step 1: compare shapes. $A$ and $B^{\mathsf T}$ both have shape $(2,3)$.

Step 2: identify the operator. * is elementwise multiplication, so it multiplies the six aligned pairs and keeps shape $(2,3)$. It does not pair a row with a column and sum over three terms.

Step 3: contrast the matrix product. A @ B uses shapes $(2,3)$ and $(3,2)$, contracts the shared dimension, and returns $(2,2)$. For these values, $A*B^{\mathsf T}=\begin{bmatrix}2&-3&0\\2&0&4\end{bmatrix}$, whereas $AB=\begin{bmatrix}-1&10\\4&6\end{bmatrix}$.

Question 7

A has shape (2, 3) and D has shape (2, 2). Why does A @ D fail, and is D @ A valid?

Show the step-by-step solution

Step 1: test $A D$. The inner dimensions would be 3 and 2. They differ, so no row of $A$ can be paired with a complete column of $D$ using the matrix-product rule; A @ D is invalid.

Step 2: test $D A$. The inner dimensions are now 2 and 2, so the product is valid and has outer shape $(2,3)$.

Step 3: preserve the order. A reversal can change legality and always changes the map being represented. It is not a general repair to swap factors until a shape happens to fit.

Question 8

For P = [[1, −1], [0, 0]] and Q = [[3, 3], [3, 3]], why can P Q be the zero matrix even though the product is valid? What does I₂ C do?

Show the step-by-step solution

Step 1: check the product. Both matrices are $(2,2)$, so the inner dimensions match.

Step 2: explain the zero. Each entry in the first output row is $1\cdot3+(-1)\cdot3=0$, and the second row contributes zero directly. Thus $PQ$ is zero because of cancellation and zero entries, not because the shapes were incompatible.

Step 3: apply the identity. With $I_2=\begin{bmatrix}1&0\\0&1\end{bmatrix}$ and $C$ shape $(2,2)$, $I_2C=C$. The identity preserves a compatible product; it does not change the values or fix a mismatch.

Question 9

For torch.nn.Linear(3, 2), a batch has shape (5, 3). State the weight shape, bias shape, output shape, and row-batch formula.

Show the step-by-step solution

Step 1: read the layer parameters. in_features=3 and out_features=2 mean that the learned weight has shape $(2,3)$ and the bias has shape $(2,)$.

Step 2: transpose for row inputs. A batch of five rows has shape $(5,3)$, so multiplying by $W^{\mathsf T}$ uses $(5,3)(3,2)$.

Step 3: state the result. The output has shape $(5,2)$ and the formula is $Y=XW^{\mathsf T}+b$. The five remains the batch dimension; each row receives two output features and the two-entry bias is added to every row.

Sources

1. Mathematics for Machine Learning companion website
2. Mathematics for Machine Learning book PDF
3. NumPy documentation: numpy.matmul
4. NumPy documentation: numpy.multiply
5. NumPy quickstart: elementwise and matrix products
6. NumPy documentation: broadcasting
7. PyTorch documentation: torch.nn.Linear
8. PyTorch documentation: torch.matmul