Matrices Represent Linear Mappings

A model exposes a linear projection, but its weight matrix is hidden. Can a few carefully chosen inputs reveal the whole transformation? Yes: send in the coordinate basis vectors and record their outputs. Those responses are exactly the matrix columns, and linearity tells us how to combine them for every other input.

By the end, you will be able to derive a matrix from a mapping’s basis responses, reconstruct an output from its coordinates, and recognize how a library’s weight layout relates to the column-vector convention.

Learning outcome: Derive a mapping matrix from transformed basis vectors.

Lesson 19 showed how a matrix multiplies an input by taking weighted sums of its columns. Lesson 34 established that a linear mapping preserves combinations. Section 2.7 of the official Mathematics for Machine Learning book connects these ideas: once a basis is chosen, the mapping is determined by its action on the basis. The derivation, projection example, visual, and questions below are original.

Each matrix column records one basis response

In $\mathbb{R}^n$, the standard basis vectors are $e_1=(1,0,\ldots,0)$, $e_2=(0,1,\ldots,0)$, and so on. Any input $x=(x_1,\ldots,x_n)$ can be built from them:

$$x=x_1e_1+x_2e_2+\cdots+x_ne_n.$$

Let $T:\mathbb{R}^n\to\mathbb{R}^m$ be linear. Since it preserves sums and scaling, it must send that input to

$$T(x)=x_1T(e_1)+x_2T(e_2)+\cdots+x_nT(e_n).$$

The outputs $T(e_j)$ are vectors in $\mathbb{R}^m$. Put each one in a column to form an $m\times n$ matrix:

This construction works in any finite dimensions: $n$ input coordinates give $n$ columns, while each response has $m$ output coordinates and therefore contributes $m$ rows. The same idea works with any ordered input basis $B=(b_1,\ldots,b_n)$ and output basis $C=(c_1,\ldots,c_m)$. Write the response to each input basis vector in output coordinates,

$$T(b_j)=a_{1j}c_1+\cdots+a_{mj}c_m.$$

The coefficients become column $j$ of the transformation matrix $A_{C\leftarrow B}$:

$$A_{C\leftarrow B}=\begin{bmatrix}[T(b_1)]_C&\cdots&[T(b_n)]_C\end{bmatrix}.$$

If $[x]_B$ is the coordinate vector of $x$ in basis $B$, then $[T(x)]_C=A_{C\leftarrow B}[x]_B$. The matrix therefore depends on the chosen coordinate systems, even though the underlying mapping $T$ does not.

• T(e₁) = (2, 1) — column 1
• T(e₂) = (1, 1) — column 2

So A is2  11  1. Every inputx = (x₁,x₂) maps tox₁(2, 1) + x₂(1, 1).

The visual uses $T(e_1)=(2,1)$ and $T(e_2)=(1,1)$. Each input arrow becomes a column, and the unit square turns into a parallelogram spanned by those two columns. The text version of the transformation is $A=\begin{bmatrix}2&1\\1&1\end{bmatrix}$, so the conclusion remains clear without motion or color.

Recover the rule from two responses

Suppose a linear map sends the two standard basis vectors to

The order matters: if the responses are reversed, the columns are reversed and so is the represented mapping. Rows instead answer a different question: each row lists how one output coordinate combines all the input coordinates.

Probe a learned projection with NumPy

The same idea can inspect a linear projection even when its weights are not directly available, provided the projection accepts chosen inputs and is actually linear. Suppose the three basis inputs produce $(1,2)$, $(-2,1)$, and $(0,3)$. Their coordinates are outputs in $\mathbb{R}^2$, so the recovered map goes from three input features to two output features. NumPy’s column_stack puts one-dimensional responses into matrix columns (NumPy documentation):

import numpy as np

responses = [
    np.array([1.0, 2.0]),
    np.array([-2.0, 1.0]),
    np.array([0.0, 3.0]),
]
A = np.column_stack(responses)  # shape (2 outputs, 3 inputs)
x = np.array([2.0, -1.0, 1.0])

print(A)
# [[ 1. -2.  0.]
#  [ 2.  1.  3.]]
print(A @ x)
# [4. 6.]

The output is $2(1,2)-(-2,1)+(0,3)=(4,6)$. For a batch of column-vector inputs, multiply by $A$ on the left; if examples are stored as rows, the same operation is written with the transpose on the right.

That orientation is useful when reading neural-network parameters. PyTorch’s torch.nn.Linear stores weights with shape (out_features, in_features) and applies a row-oriented batch as $XW^\mathsf{T}+b$ (PyTorch documentation). With bias=False, its weight tensor $W$ is the same matrix $A$ in the column-vector equation $T(x)=Ax$. If bias is enabled, the module instead maps $x$ to $Ax+b$: subtract the zero-input response $b$ before treating basis outputs as columns. If a nonlinear operation such as an activation follows the layer, the overall mapping is no longer linear, so the basis responses of the layer alone do not determine its behavior for every input.

The basis responses are enough only when the map is linear

The reconstruction has two ingredients: a basis lets every input be expressed as a weighted sum, and linearity lets the mapping pass those weights through the sum. Together, they show both why the matrix reproduces $T$ and why no second matrix can produce a different output while matching all the same basis responses.

If a system returns $T(e_j)$ for every input basis vector, those responses determine its matrix. But testing those inputs does not prove an unknown system is linear. For example, a nonlinear system could match the same basis responses and behave differently at $e_1+e_2$. A held-out combination tests that assumption: compare $T(e_1+e_2)$ with $T(e_1)+T(e_2)$. The next lesson uses matrices to ask a further question: which input directions disappear, and which outputs can the map reach?

Check your understanding

Question 1

A linear map sends e₁ to (4, −1) and e₂ to (2, 3). Write its matrix in standard coordinates and identify the input and output dimensions.

Show the step-by-step solution

The two inputs form the standard basis of $\mathbb{R}^2$, so there are two columns. Each output has two coordinates, so there are two rows. Place the first response in column one and the second in column two:

$$A=\begin{bmatrix}4&2\\-1&3\end{bmatrix}.$$

Thus the map is $T:\mathbb{R}^2\to\mathbb{R}^2$.

Question 2

Using the map from the worked example, compute T(1, 2) from its basis responses. Show how the coordinates determine the combination.

Show the step-by-step solution

The input decomposes as $(1,2)=1e_1+2e_2$. Linearity carries those coefficients to the outputs:

$$T(1,2)=T(e_1)+2T(e_2) =(3,1)+2(-1,2) =(3,1)+(-2,4) =(1,5).$$

The same check with the matrix is $\begin{bmatrix}3&-1\\1&2\end{bmatrix}\begin{bmatrix}1\\2\end{bmatrix} =\begin{bmatrix}1\\5\end{bmatrix}$.

Question 3

A map from ℝ³ to ℝ² sends e₁ to (1, 0), e₂ to (0, 2), and e₃ to (−1, 1). Construct its matrix and compute the image of (2, −1, 3).

Show the step-by-step solution

There are three input basis vectors, so place their responses in three columns:

$$A=\begin{bmatrix}1&0&-1\\0&2&1\end{bmatrix}.$$

Apply this matrix to the input, or combine the responses with the input coordinates:

$$T(2,-1,3)=2(1,0)-1(0,2)+3(-1,1) =(2,0)+(0,-2)+(-3,3)=(-1,1).$$

The output has two coordinates, as expected for a map into $\mathbb{R}^2$.

Question 4

The responses T(e₁) = (2, 1) and T(e₂) = (−1, 3) are placed as rows instead of columns. What matrix results, and what does it do to (1, 0)?

Show the step-by-step solution

Putting the listed responses as rows gives $B=\begin{bmatrix}2&1\\-1&3\end{bmatrix}$. Multiplying by $(1,0)$ selects the first column, so $B(1,0)=(2,-1)$. The actual response should be $T(e_1)=(2,1)$, which differs. Putting outputs in rows has changed the map; the general rule is to put $T(e_j)$ in column $j$.

Question 5

A projection module returns f(0) = (5, −2), f(e₁) = (7, 1), and f(e₂) = (4, 0). If it is affine, recover its linear matrix part and its bias vector.

Show the step-by-step solution

For an affine map $f(x)=Ax+b$, the zero input gives $f(0)=b$, so $b=(5,-2)$. The first linear response is the first basis output after subtracting that bias: $f(e_1)-b=(7,1)-(5,-2)=(2,3)$. Similarly, $f(e_2)-b=(4,0)-(5,-2)=(-1,2)$. Put those corrected responses in columns:

$$A=\begin{bmatrix}2&-1\\3&2\end{bmatrix},\qquad b=\begin{bmatrix}5\\-2\end{bmatrix}.$$

Question 6

Why do the responses to the standard basis determine T(x) for every x when T is linear? State what assumption makes the conclusion possible.

Show the step-by-step solution

Every $x=(x_1,\ldots,x_n)$ has the decomposition $x=\sum_{j=1}^n x_j e_j$. By linearity, $T(x)=T(\sum_j x_j e_j)=\sum_j x_jT(e_j)$. The recorded basis responses therefore supply every term needed to calculate any output. The conclusion depends on $T$ preserving sums and scalar multiplication; without linearity, other inputs can behave differently even when all basis responses match.

Question 7

A black-box function matches a linear map on every standard basis vector. Does that establish the function is linear? Give one additional test in two dimensions.

Show the step-by-step solution

No. A finite set of matching responses does not rule out nonlinear behavior elsewhere. In two dimensions, test an input combination such as $e_1+e_2$: linearity requires $T(e_1+e_2)=T(e_1)+T(e_2)$. A mismatch disproves linearity. A match is useful evidence for that particular combination, but alone it does not prove the rule for every possible input.

Sources

1. Mathematics for Machine Learning book PDF, Section 2.7
2. NumPy documentation: numpy.column_stack
3. PyTorch documentation: torch.nn.Linear