Matrices for AI: Read Rows, Columns, and Neural Network Shapes

A dense layer accepts three features and produces two output values. Its six weights could be stored in one long list, but that list would hide the most useful debugging fact: three weights belong to each output rule. Arrange the same numbers in two rows and three columns, and the structure becomes visible:

$$W= \begin{bmatrix} 0.8 & -0.4 & 0.2\\ 0.1 & 0.6 & -0.5 \end{bmatrix}.$$

The matrix’s shape, $2\times3$, is a compact contract. The first number counts the output rules; the second counts the input features used by every rule. Before multiplying anything, the grid already tells us what each axis must mean and where every weight belongs.

By the end of this lesson, you will be able to read matrix dimensions, locate an entry from its two indices, interpret rows and columns in a dense-layer weight matrix, reproduce those inspections in NumPy, and diagnose arrays whose values look plausible but whose axes have the wrong jobs.

Learning outcome: Read matrix dimensions and interpret rows and columns.

Lesson 14: Vectors as AI Data established that an ordered list of coordinates can represent one AI input. A matrix adds a second axis so several related lists can keep their shared organization. The official Mathematics for Machine Learning companion defines matrix notation, rows, columns, and entries in Section 2.2 of the book. The dense-layer scenario, numbers, diagram, code, explanations, and assessments below are original.

A rectangle carries two separate counts

The two indices have different jobs. The first index moves vertically between rows; the second moves horizontally between columns. For

$$B= \begin{bmatrix} 7 & 4 & 9\\ 2 & 5 & 8 \end{bmatrix},$$

$B\in\mathbb{R}^{2\times3}$, $b_{12}=4$, and $b_{23}=8$. The matrix has $2\cdot3=6$ entries. Adding the dimensions, $2+3$, happens to produce the same number here, but that is a coincidence: a $3\times4$ matrix has $12$ entries, not $7$.

A matrix need not be square. A $2\times3$ matrix is as legitimate as a $3\times3$ matrix. Nor does the notation decide what an axis represents. Rows might be samples in a dataset, output rules in a layer, or time steps in a sequence. The data contract supplies that meaning.

Two output rules become a 2 × 3 matrix

Suppose a small ranking layer reads three input features:

1. $x_1$: topic match;
2. $x_2$: recency; and
3. $x_3$: source quality.

It stores one coefficient for each feature in each of two output rules:

$$\begin{aligned} r_1(\boldsymbol{x}) &= 0.8x_1-0.4x_2+0.2x_3,\\ r_2(\boldsymbol{x}) &= 0.1x_1+0.6x_2-0.5x_3. \end{aligned}$$

The first rule will produce a relevance value; the second will produce a review-priority value. This lesson is not yet computing those values. The task is to organize the coefficients without losing which input and output each one connects.

The construction works because the row and column labels remain stable. If the feature order changed to [source quality, topic match, recency] while the numbers stayed put, the matrix would retain the same shape and values but describe different rules.

Semantic alternative: the two output rules stored in the 2 × 3 weight matrix

Output rule (row)	Topic match (column 1)	Recency (column 2)	Source quality (column 3)	Row interpretation
row 1 · relevance	0.8 (w₁₁)	−0.4 (w₁₂)	0.2 (w₁₃)	Three weights associated with the relevance output.
row 2 · review priority	0.1 (w₂₁)	0.6 (w₂₂)	★ −0.5 (w₂₃)	Three weights associated with the review-priority output; w₂₃ is −0.5.

The outlined entry in the visual is identified in two independent ways: the star and dashed outline mark it visually, while the label and table state $w_{23}=-0.5$ directly. Row 2 chooses the review-priority rule. Column 3 chooses source quality. Their intersection identifies one relationship in the layer.

A cell records a row–column relationship

An entry is more than a loose number. Under the convention above, $w_{ij}$ answers a precise question:

Which weight connects input feature $j$ to output rule $i$?

The answer follows from the indices rather than from color or position alone. For example, $w_{21}=0.1$ belongs to output rule 2 and input feature 1. The entire second row,

$$\begin{bmatrix}0.1&0.6&-0.5\end{bmatrix},$$

stores the coefficients for one output. The entire second column,

$$\begin{bmatrix}-0.4\\0.6\end{bmatrix},$$

collects the recency coefficient from both outputs.

That row interpretation belongs to this weight convention; it is not a universal law that every matrix row means “output.” A dataset matrix may put one observation in each row instead. Always name the axis contract before reasoning from a shape.

NumPy makes both axes inspectable

NumPy represents this grid as a two-dimensional ndarray. Its shape property returns a tuple of axis lengths. Basic ndarray indexing is zero-based, so Python position 0 corresponds to mathematical index $1$.

import numpy as np

weights = np.array([
    [0.8, -0.4,  0.2],
    [0.1,  0.6, -0.5],
], dtype=float)

print("shape:", weights.shape)
print("number of axes:", weights.ndim)
print("number of entries:", weights.size)
print("row 2:", weights[1, :])
print("column 3:", weights[:, 2])
print("w_23:", weights[1, 2])

Verified output:

shape: (2, 3)
number of axes: 2
number of entries: 6
row 2: [ 0.1  0.6 -0.5]
column 3: [ 0.2 -0.5]
w_23: -0.5

The punctuation in the index matters:

Intended selection	Mathematical description	NumPy expression	Result
One entry	row 2, column 3	weights[1, 2]	-0.5
One row	every column in row 2	weights[1, :]	[0.1, 0.6, -0.5]
One column	every row in column 3	weights[:, 2]	[0.2, -0.5]

The colon means “take every position along this axis.” A single integer fixes one axis. This makes a useful debugging sequence: inspect the whole shape, then a row, then a column, and finally the exact entry whose meaning is in question.

Shape checks should encode the model contract rather than merely print it:

expected_shape = (2, 3)  # two outputs, three input features

if weights.shape != expected_shape:
    raise ValueError(
        f"expected {expected_shape}, got {weights.shape}; "
        "check the output and input axes"
    )

The check can catch a transposed array. It cannot catch a silent feature-order change because [topic match, recency, source quality] and [recency, source quality, topic match] both have length three. Names and provenance must protect the semantic contract.

A dense layer exposes its input and output contract

When an input is written as a column vector $\boldsymbol{x}\in\mathbb{R}^{3}$, the layer can be summarized as

$$\boldsymbol{y}=W\boldsymbol{x}+\boldsymbol{b}, \qquad W\in\mathbb{R}^{2\times3}, \quad \boldsymbol{b}\in\mathbb{R}^{2}, \quad \boldsymbol{y}\in\mathbb{R}^{2}.$$

The neighboring sizes foreshadow the next operation:

$$(2\times3)(3\times1)\longrightarrow(2\times1).$$

Three input coordinates fit the three columns of $W$; the two rows produce two output coordinates. Lesson 19 will calculate that matrix–vector product as weighted sums. For now, the dimensions alone reveal whether the proposed objects can fit this convention.

The current torch.nn.Linear documentation uses the same stored-weight layout: a layer with in_features=3 and out_features=2 has a weight tensor shaped (out_features, in_features), or (2, 3). PyTorch commonly writes batched inputs with features on the final axis and expresses its operation as $\boldsymbol{y}=\boldsymbol{x}A^{\mathsf T}+\boldsymbol{b}$. That row-oriented batch notation explains the visible transpose; it does not change the stored weight’s two-row-by-three-column contract.

The bias is not a third matrix column under this definition. It is a separate two-coordinate vector, one offset per output. A bias-using layer therefore has six weight scalars plus two bias scalars in this example.

Three shape shortcuts produce the wrong mental model

There is also a software boundary that ordinary classroom notation often leaves implicit. This lesson defined $m,n\ge1$, so every matrix has at least one row and one column. NumPy can nevertheless construct an array with shape (0, 3) or (2, 0). Such an array has two axes and zero entries, but it cannot serve as the two-output, three-input weight grid above. A framework’s shape rules and the mathematical model’s meaning must both pass.

The rows are ready for the next operation

A matrix earns its compactness by keeping two promises at once. Its first axis preserves the collection of output rules; its second aligns every rule to the same ordered input features. That organization makes $w_{ij}$ readable before arithmetic begins and makes a wrong transpose visible before it reaches a training run.

The next prerequisite-safe lesson will put an input vector beside this grid. Each row will then form one weighted sum, turning the $2\times3$ organization into a two-coordinate result. The Math for AI course page keeps the ordered lessons and their prerequisites together.

Check your understanding

Question 1

Let B = [[3, −1, 4], [0, 5, 2]]. State its dimensions, total number of entries, b₁₃, and b₂₁.

Show the step-by-step solution

Count rows first: $B$ has two horizontal rows, so $m=2$. Each row has three entries, so $n=3$. Therefore

$$B\in\mathbb{R}^{2\times3} \quad\text{and}\quad mn=2\cdot3=6.$$

For $b_{13}$, choose row 1 and then column 3; the value there is $4$. For $b_{21}$, choose row 2 and then column 1; the value there is $0$. Thus $b_{13}=4$ and $b_{21}=0$. The index order is row first, column second.

Question 2

Two output rules use inputs x₁, x₂, and x₃: s₁ = 2x₁ + 0x₂ − x₃ and s₂ = 0.5x₁ + 3x₂ + 4x₃. Build the matrix with one rule per row and verify two entries.

Show the step-by-step solution

There are two output rules, so the matrix needs two rows. Each rule uses three ordered inputs, so it needs three columns. Keep the coefficients of each rule in their $x_1,x_2,x_3$ order:

$$S= \begin{bmatrix} 2 & 0 & -1\\ 0.5 & 3 & 4 \end{bmatrix} \in\mathbb{R}^{2\times3}.$$

The coefficient of $x_3$ in rule 1 is at row 1, column 3, so $s_{13}=-1$. The coefficient of $x_2$ in rule 2 is at row 2, column 2, so $s_{22}=3$. Those checks agree with the two original rules.

Question 3

For the lesson's weight matrix W, which NumPy expressions select mathematical entry w₂₃, all of row 2, and all of column 3? Explain the off-by-one shift.

Show the step-by-step solution

Mathematical indices in the lesson begin at $1$, while NumPy positions begin at 0. Subtract one from each mathematical index when writing a direct NumPy selection:

• $w_{23}$ becomes weights[1, 2] and returns -0.5;
• row 2 becomes weights[1, :] and returns [0.1, 0.6, -0.5]; and
• column 3 becomes weights[:, 2] and returns [0.2, -0.5].

The colon leaves that axis unrestricted. The integer fixes the other axis at one zero-based position.

Question 4

A dense layer has five input features, four output units, and one bias per output. What is the weight-matrix shape, and how many trainable scalars does the layer contain?

Show the step-by-step solution

Using the (out_features, in_features) convention, four output units supply the rows and five input features supply the columns:

$$W\in\mathbb{R}^{4\times5}.$$

The weight matrix contains $4\cdot5=20$ scalars. The bias vector has one entry for each of the four outputs, so it contributes $4$ more. The total is

$$20+4=24$$

trainable scalars. The bias count is added separately; it does not change the matrix from $4\times5$ to $4\times6$.

Question 5

A developer intends a three-input, two-output layer but finds weights.shape == (3, 2). Diagnose the axis mistake and state a safe next step.

Show the step-by-step solution

The intended stored layout is (out_features, in_features). With two outputs and three inputs, the expected shape is (2, 3), not (3, 2).

The observed shape has put the three input-feature groups on the row axis and the two output groups on the column axis. If the values were deliberately built with one output rule per column, weights.T would produce shape (2, 3). But transposing blindly is unsafe: first verify the feature and output labels, then transpose only if those labels confirm that the axes were intentionally reversed. Shape repair must preserve semantic meaning.

Question 6

Matrices C = [[1, 2, 3, 4]] and D = [[1], [2], [3], [4]] contain the same four values. Why are they not interchangeable?

Show the step-by-step solution

$C$ has one row and four columns, so $C\in\mathbb{R}^{1\times4}$. It can store one rule that uses four ordered inputs. $D$ has four rows and one column, so $D\in\mathbb{R}^{4\times1}$. It can store four one-entry rows or represent a column vector.

Their entry counts both equal $4$, but their axes have different lengths and can carry different meanings. Transposition relates them, $D=C^{\mathsf T}$, yet that operation switches rows and columns. Equal values and equal entry counts do not imply equal matrix shapes or contracts.

Question 7

NumPy can create np.empty((0, 3)). What are its shape and size, and why can it not replace the lesson's two-output, three-input weight matrix?

Show the step-by-step solution

The tuple passed to np.empty is the array’s shape, so the result has shape (0, 3): zero rows and three columns. Its number of entries is the product of the axis lengths,

$$0\cdot3=0.$$

The lesson’s layer requires two output rules, hence two rows. The empty array stores none. Its three-column axis happens to match the input-feature count, but the missing output axis violates the model contract. A shape-valid NumPy array is not automatically a useful matrix for a particular layer.

Question 8

In the matrix map, w₂₃ = −0.5. State the relationship it names. Does that one negative weight prove that the final review-priority output will be negative?

Show the step-by-step solution

Read the indices in order. The first index, $2$, selects row 2: the review-priority output rule. The second index, $3$, selects column 3: the source-quality input feature. Therefore $w_{23}=-0.5$ is the coefficient that connects source quality to the review-priority rule.

That one coefficient does not determine the final output’s sign. The output will also depend on the actual value of $x_3$, the other two weighted input contributions, and any bias. The matrix entry names one relationship; the next lesson’s weighted-sum calculation combines all contributions.

Sources

1. Mathematics for Machine Learning companion website
2. Mathematics for Machine Learning book PDF
3. NumPy documentation: ndarray.shape
4. NumPy documentation: indexing on ndarrays
5. PyTorch documentation: torch.nn.Linear