A Basis Gives AI Vectors a Reversible Coordinate System

The same point in a model’s two-dimensional representation can be written as (1, 7) or (3, 2). Both pairs can describe the vector whose first coordinate is 1 and second is 7, provided we say which coordinate system we are using. The first pair uses the familiar horizontal and vertical axes. The second uses two slanted basis vectors. A basis is what makes that alternative a complete, non-ambiguous way to describe every vector in the space.

By the end, you will be able to check whether a set of vectors is a basis, calculate a vector’s coordinates in that basis, and explain what must change in a linear model when its input coordinates change.

Learning outcome: Verify a basis and express a vector in basis coordinates.

This lesson builds on linear independence and span: a basis combines both ideas. The treatment follows Section 2.6 of Mathematics for Machine Learning, while the numerical example, model-score calculation, explorer, and questions below are original.

A basis has just enough directions

Work in a finite-dimensional vector space. A set of vectors is a basis for a space when it satisfies two tests:

It spans the space: every vector in the space can be built from the set using a linear combination.
It is linearly independent: none of its vectors is redundant; the only linear combination that produces the zero vector uses zero for every coefficient.

Together, these conditions mean the set has enough directions to reach every vector and has no extra direction that creates ambiguity. The book’s Section 2.6 gives equivalent characterizations: a basis is a minimal spanning set and a maximal linearly independent set, and each vector has a unique linear combination in that basis.

In $\mathbb{R}^n$ , a basis has exactly $n$ vectors. For a smaller subspace, the basis has as many vectors as the subspace has independent directions. Two vectors may be a basis for a plane inside $\mathbb{R}^3$ , for example, without being a basis for all of $\mathbb{R}^3$ .

Why coordinates become unique

Suppose $\boldsymbol{b}_1,\ldots,\boldsymbol{b}_n$ are a basis and $\boldsymbol{v}$ is any vector in the space. Spanning guarantees that at least one set of coefficients can represent it:

\boldsymbol{v}=c_1\boldsymbol{b}_1+\cdots+c_n\boldsymbol{b}_n.

Now imagine another set $d_1,\ldots,d_n$ gives the same vector. Subtracting the two expressions gives

0=(c_1-d_1)\boldsymbol{b}_1+\cdots+(c_n-d_n)\boldsymbol{b}_n.

Because the basis vectors are linearly independent, every coefficient in this zero combination must be zero. So $c_i=d_i$ for each $i$ . The coordinates are unique because the basis has no redundant direction.

We write the coordinate column as $[\boldsymbol{v}]_B$ . The subscript matters: it says which basis gives those coordinates. The vector and its coordinate column are related, but they are not interchangeable objects.

Compute a change of coordinates

Take these two vectors in $\mathbb{R}^2$ :

\boldsymbol{b}_1=\begin{bmatrix}1\\1\end{bmatrix},\qquad \boldsymbol{b}_2=\begin{bmatrix}-1\\2\end{bmatrix}.

Put the basis vectors in the columns of a matrix:

B=\begin{bmatrix}1&-1\\1&2\end{bmatrix}.

They are independent: the determinant is $1\cdot2-(-1\cdot1)=3$ , which is not zero. Since there are two independent vectors in $\mathbb{R}^2$ , they also span the plane, so they form a basis.

Find the coordinates of $\boldsymbol{v}=(1,7)^\mathsf{T}$ . The coordinate column $\boldsymbol{c}=[\boldsymbol{v}]_B$ must satisfy $B\boldsymbol{c}=\boldsymbol{v}$ . Writing $\boldsymbol{c}=(c_1,c_2)^\mathsf{T}$ turns that matrix equation into two ordinary equations:

\begin{aligned} c_1-c_2&=1,\\ c_1+2c_2&=7. \end{aligned}

Subtract the first equation from the second: $3c_2=6$ , so $c_2=2$ . Then $c_1-2=1$ , so $c_1=3$ . Check the reconstruction directly:

3\begin{bmatrix}1\\1\end{bmatrix} +2\begin{bmatrix}-1\\2\end{bmatrix} =\begin{bmatrix}3-2\\3+4\end{bmatrix} =\begin{bmatrix}1\\7\end{bmatrix}.

The coordinate pair $(3,2)$ means “three steps along $\boldsymbol{b}_1$ and two along $\boldsymbol{b}_2$ .” It names the same endpoint that standard coordinates call $(1,7)$ .

One vector, two coordinate systems

Choose a coordinate system

Coordinates in the selected basis: (1, 7)

Vector in standard coordinates: (1, 7)

v = c₁b₁ + c₂b₂

Coordinates in the selected basis: (1, 7)

The endpoint stays fixed at (1, 7). Changing the basis changes the coordinate pair used to name that point.

The explorer starts with standard coordinates. Use its keyboard-accessible selector to switch to the tilted basis: the arrow to $\boldsymbol{v}$ ends at the same point, while the displayed coordinates change from $(1,7)$ to $(3,2)$ . The caption and readout expose the result as text as well as in the diagram.

Let NumPy solve the coordinate equation

For a larger basis, the same relationship is a linear system. Store basis vectors as columns, then solve $B\boldsymbol{c}=\boldsymbol{v}$ :

import numpy as np

B = np.array([[1, -1],
              [1,  2]])  # columns are b1=(1, 1) and b2=(-1, 2)
v = np.array([1, 7])

rank = np.linalg.matrix_rank(B)
coordinates = np.linalg.solve(B, v)
reconstructed = B @ coordinates

print(rank)                         # 2: full rank in R^2
print(coordinates)                  # [3. 2.]
print(np.allclose(reconstructed, v)) # True

For this square matrix in $\mathbb{R}^2$ , full rank means its two columns are independent, which is enough to establish a basis. numpy.linalg.solve returns the solution of the square, full-rank system; NumPy raises LinAlgError when the matrix is singular or not square. The rank result here is a numerical diagnostic: NumPy estimates rank through singular values and a tolerance, so near-dependent floating-point data may need a tolerance chosen for the scale and measurement precision of the application. See the official NumPy matrix_rank reference and solve reference.

For exact small integer examples, a hand calculation such as the determinant above gives a crisp proof. For real-valued data, a reconstruction check is useful but is not a basis test on its own: a singular matrix can still reconstruct some vectors. Check that the matrix has the right shape and full rank as well.

A model’s weights must follow its coordinates

A basis change can clarify a model’s internal representation, but the change has to be coordinated with the operations that consume it. Consider a linear score $s=\boldsymbol{w}^{\mathsf T}\boldsymbol{v}$ with $\boldsymbol{w}=(2,1)^\mathsf{T}$ and $\boldsymbol{v}=(1,7)^\mathsf{T}$ . In standard coordinates the score is

2\cdot1+1\cdot7=9.

In the tilted basis, $\boldsymbol{v}=B\boldsymbol{c}$ with $\boldsymbol{c}=(3,2)^\mathsf{T}$ . Then

\boldsymbol{w}^{\mathsf T}\boldsymbol{v} =\boldsymbol{w}^{\mathsf T}B\boldsymbol{c} =(B^{\mathsf T}\boldsymbol{w})^{\mathsf T}\boldsymbol{c}.

The matching weights in the new coordinates are $B^{\mathsf T}\boldsymbol{w}=(3,0)^\mathsf{T}$ , and the score remains $3\cdot3+0\cdot2=9$ . If the original weights $(2,1)$ were applied directly to the new coordinate pair, the score would become $2\cdot3+1\cdot2=8$ . The representation was re-described, but the model’s parameters were not updated to match.

This equality is exact for the linear score shown. A neural network that applies nonlinear activations, normalization, or other operations between layers may not preserve its behavior under an arbitrary basis change. In practice, a representation’s coordinates are tied to the learned parameters and operations around it; changing one piece alone can change predictions.

When a candidate set fails

Two candidate vectors can point in the same direction and still look like two entries in a list:

\boldsymbol{a}_1=\begin{bmatrix}1\\1\end{bmatrix},\qquad \boldsymbol{a}_2=\begin{bmatrix}2\\2\end{bmatrix}.

The second is twice the first. Their determinant is zero, and their span is only the line $y=x$ , not the whole plane. The vector $(1,7)$ cannot be reached at all using them. Even a reachable vector has ambiguous coordinates: $(2,2)$ can be written as $2\boldsymbol{a}_1+0\boldsymbol{a}_2$ or $0\boldsymbol{a}_1+1\boldsymbol{a}_2$ .

Nearly parallel vectors can pass an exact independence test yet produce coordinates that are sensitive to small measurement or rounding changes. That is a numerical conditioning issue: the mathematical coordinates may be unique while their computed values are fragile. A rank estimate and a reconstruction residual answer different questions, so interpret both in the context of data precision.

A basis is therefore more than a compact list of vectors. It is a contract: every vector in the chosen space is reachable, and only one coordinate tuple describes each vector once the basis is fixed. The number of basis vectors then gives the space’s dimension, the next step in connecting linear algebra to model representations.

Check your understanding

Question 1

Do (2, 0) and (1, 3) form a basis for ℝ²? Show the independence check and explain why it is sufficient here.