Sigma Notation for AI: How Losses Become One Training Objective

A model makes four predictions and produces four squared losses: $1$ , $4$ , $1$ , and $9$ . A training step needs one objective, not four loose numbers. Adding the losses gives $15$ ; averaging them gives $3.75$ . Both are valid reductions, but they answer different questions and lead to different scales in code.

Sigma notation turns that repeated addition into one compact expression. By the end of this lesson, you will be able to expand a sum or product from its bounds, translate it into a loop, compute mean squared error (MSE) in NumPy, and catch reductions that use the wrong bounds, denominator, or axis.

This lesson builds on Lesson 5’s indexed values: once $\ell_i$ means “the loss for example $i$ ,” the next task is to combine all the indexed losses. The official Mathematics for Machine Learning companion is the source home for the course’s reference book; its introductory chapter motivates the route from numerical data to models and learning objectives.

The bounds tell a sum where to start and stop

The Greek capital letter sigma, $\sum$ , means “add a sequence of terms.” For example,

\sum_{i=2}^{5} a_i

has four parts to read:

$i$ is the index that changes;
$2$ is the lower bound, where $i$ starts;
$5$ is the upper bound, where $i$ stops; and
$a_i$ is the term to evaluate at each index.

Both bounds are included, so the expansion is

\sum_{i=2}^{5} a_i=a_2+a_3+a_4+a_5.

The expansion is also an algorithm. A running total begins at zero and absorbs one term at a time:

a = [10, 20, 30, 40, 50]
n = 5
total = 0

for i in range(1, n + 1):
    total += a[i - 1]

print(total)

The mathematical index runs from $1$ through $n$ , while the Python list uses positions $0$ through $n-1$ . Python’s stop value is excluded, so range(1, n + 1) is required to include $n$ . Writing range(1, n) silently omits the final term. The expression a[i - 1] translates the one-based mathematical index into the zero-based list position introduced in Lesson 5.

Product notation changes the starting value and operation

The Greek capital letter pi, $\prod$ , means “multiply a sequence of terms.” It uses the same kind of index and inclusive bounds:

\prod_{i=2}^{5} a_i=a_2a_3a_4a_5.

For $a_2=2$ , $a_3=3$ , $a_4=1$ , and $a_5=4$ ,

\prod_{i=2}^{5}a_i=2\times3\times1\times4=24.

The corresponding loop starts at $1$ , because multiplying by $1$ leaves a number unchanged:

values = [2, 3, 1, 4]
product = 1

for value in values:
    product *= value

print(product)

Starting product at 0 would make every later result zero. This difference also explains the standard empty cases. A sum with no terms is $0$ and a product with no terms is $1$ : each uses the neutral starting value that leaves later additions or multiplications unchanged. NumPy documents the same conventions for np.sum and np.prod.

Repeated operation	Notation	Accumulator starts at	Empty result
Addition	$\sum$	$0$	$0$
Multiplication	$\prod$	$1$	$1$

Products will become especially important when the course reaches joint probabilities. For now, the key implementation habit is to match the accumulator and its starting value to the notation.

Four prediction errors become one mean squared error

Suppose a regression model predicts one number for each of four examples:

\hat{\boldsymbol{y}}=(2,0,4,5), \qquad \boldsymbol{y}=(1,2,5,2).

Here $\hat y_i$ is the prediction for example $i$ , and $y_i$ is its target. Define the per-example squared loss as

\ell_i=(\hat y_i-y_i)^2.

Compute MSE without skipping the individual losses

First calculate each prediction error:

\begin{aligned} \hat y_1-y_1 &= 2-1=1,\\ \hat y_2-y_2 &= 0-2=-2,\\ \hat y_3-y_3 &= 4-5=-1,\\ \hat y_4-y_4 &= 5-2=3. \end{aligned}

Then square each error:

(\ell_1,\ell_2,\ell_3,\ell_4)=(1^2,(-2)^2,(-1)^2,3^2)=(1,4,1,9).

The sum of squared errors is

\sum_{i=1}^{4}\ell_i=1+4+1+9=15.

MSE divides that sum by the number of examples, $n=4$ :

\operatorname{MSE} =\frac{1}{n}\sum_{i=1}^{n}(\hat y_i-y_i)^2 =\frac{15}{4} =3.75.

The interactive accumulator exposes the intermediate state that sigma notation compresses. Its table remains a complete text alternative: row $i$ lists $\hat y_i$ , $y_i$ , the squared loss, and whether that loss has entered the running total.

Build the training objective one example at a time. Play the sequence or move through it with the buttons.

Predictions, targets, squared losses, and their accumulator state
Example	Prediction	Target	Squared loss	Accumulator state
1	2	1	1	Current addition
2	0	2	4	Waiting
3	4	5	1	Waiting
4	5	2	9	Waiting

Running sum1

Terms included: 1
Running mean: 1 ÷ 1 = 1.00

Step 1 of 4: the partial sum is 1.

After all four rows enter the accumulator, the running sum is $15$ . Dividing by the four included rows gives the MSE, $3.75$ . The animation changes no mathematics; it makes the repeated update total = total + loss visible.

NumPy mirrors the formula

NumPy can keep the per-example losses visible before reducing them:

import numpy as np

y_true = np.array([1.0, 2.0, 5.0, 2.0])
y_pred = np.array([2.0, 0.0, 4.0, 5.0])

squared_losses = (y_pred - y_true) ** 2
total_squared_error = np.sum(squared_losses)
mse = np.mean(squared_losses)

print("per-example squared losses:", squared_losses)
print("sum:", total_squared_error)
print("mean squared error:", mse)

per-example squared losses: [1. 4. 1. 9.]
sum: 15.0
mean squared error: 3.75

The np.sum documentation defines a reduction over array elements. The np.mean documentation defines the arithmetic mean and says that, by default, it uses the flattened array. In this one-dimensional example, flattening changes nothing: all four entries are reduced.

Keeping squared_losses as a named array is useful for debugging. It reveals whether one example dominates the objective and lets a test compare the implementation with the four hand calculations above.

An axis decides which indices disappear

With a multidimensional loss array, “take the mean” is incomplete until the axis is clear. Consider two examples, each with three output losses:

L= \begin{bmatrix} 1 & 4 & 1\\ 9 & 0 & 4 \end{bmatrix}.

The rows are examples and the columns are outputs. NumPy’s axis argument names the axis to reduce:

losses = np.array([
    [1.0, 4.0, 1.0],
    [9.0, 0.0, 4.0],
])

print("all six losses:", np.mean(losses))
print("one mean per example:", np.mean(losses, axis=1))
print("one mean per output:", np.mean(losses, axis=0))

all six losses: 3.1666666666666665
one mean per example: [2.         4.33333333]
one mean per output: [5.  2.  2.5]

Expression	Reduced direction	Remaining result
`np.mean(losses)`	both axes	one scalar for all six losses
`np.mean(losses, axis=1)`	columns within each row	one value for each example
`np.mean(losses, axis=0)`	rows within each column	one value for each output

Using axis=0 when the model needs one loss per example is a semantic bug. The code runs and the numbers look plausible, but the surviving index means “output,” not “example.” The direct NumPy documentation for sum and mean specifies that axis=None reduces the full array while an integer selects the axis along which the reduction is performed.

Sum and mean carry different scale

For a fixed nonempty collection of $n$ losses,

\operatorname{mean}(\ell_1,\ldots,\ell_n) =\frac{1}{n}\operatorname{sum}(\ell_1,\ldots,\ell_n).

The mean is a rescaled sum, but the scale matters when batch sizes differ. Four examples each with loss $2$ have sum $8$ and mean $2$ ; eight such examples have sum $16$ and the same mean $2$ . A sum measures the collection’s total loss. A mean measures loss per included item.

During model training, the prediction rule produces one or more losses for each example. A reduction turns those values into the scalar objective used to judge the current parameters. The choice between sum, mean, or a masked and weighted variant therefore belongs to the model’s mathematical contract, not just to code formatting.

Lesson 7 will use powers and roots to reason about model scale. Until that lesson is live, the Math for AI course page is the reliable place to continue in publication order. The notation learned here will return throughout the course: sums build weighted combinations and objectives, while products build repeated factors.

Check your understanding

Question 1

Expand the sum from i = 3 through i = 6: Σᵢ₌₃⁶ (2i − 1). Then calculate its value.

Show the step-by-step solution

The lower bound is 3 and the upper bound is 6, so both endpoints appear:

\sum_{i=3}^{6}(2i-1) =(2\cdot3-1)+(2\cdot4-1)+(2\cdot5-1)+(2\cdot6-1).

Evaluate each indexed term before adding:

5+7+9+11=32.

There are $6-3+1=4$ terms. Counting only three would reveal that an endpoint had been dropped.

Question 2

A Python loop is meant to implement Σᵢ₌₁ⁿ aᵢ but uses for i in range(1, n). What term is missing, and how should the loop index a zero-based list a?

Show the step-by-step solution

Python excludes the stop value of range, so range(1, n) produces $1,2,\ldots,n-1$ . The missing term is $a_n$ .

Include the mathematical upper bound by stopping at n + 1, then subtract 1 when addressing the zero-based list:

total = 0
for i in range(1, n + 1):
    total += a[i - 1]

Now $i=1$ reads a[0] and $i=n$ reads a[n - 1], so every term from $a_1$ through $a_n$ appears exactly once.

Question 3

Expand ∏ₖ₌₂⁴ (k + 1), compute it, and explain why its loop accumulator must start at 1.

Show the step-by-step solution

The inclusive bounds give the indices $2$ , $3$ , and $4$ :

\prod_{k=2}^{4}(k+1)=(2+1)(3+1)(4+1)=3\times4\times5=60.

A loop needs a value before it sees the first term. Starting at $1$ preserves that first multiplication because $1\times3=3$ . Starting at $0$ would instead give $0\times3\times4\times5=0$ , erasing all the terms.

Question 4

For targets (3, 1, 2) and predictions (1, 2, 2), calculate every squared loss, their sum, and the MSE.

Show the step-by-step solution

Subtract target from prediction for each example:

(1-3,\;2-1,\;2-2)=(-2,1,0).

Square those errors to obtain the per-example losses:

(\ell_1,\ell_2,\ell_3)=((-2)^2,1^2,0^2)=(4,1,0).

Their sum is $4+1+0=5$ . There are three examples, so

\operatorname{MSE}=\frac{1}{3}\sum_{i=1}^{3}\ell_i =\frac{5}{3}\approx1.667.

The decimal is an approximation; $5/3$ is the exact value.

Question 5

Batch A has losses (2, 2). Batch B has losses (2, 2, 2, 2). Compare their sums and means. Which reduction preserves the per-example scale?

Show the step-by-step solution

For Batch A,

\operatorname{sum}(A)=2+2=4, \qquad \operatorname{mean}(A)=\frac{4}{2}=2.

For Batch B,

\operatorname{sum}(B)=2+2+2+2=8, \qquad \operatorname{mean}(B)=\frac{8}{4}=2.

The sum doubles because Batch B contains twice as many examples. The mean remains $2$ , so it preserves the loss-per-example scale in this comparison.

Question 6

What are the empty sum and empty product, and what implementation bug occurs if both accumulator loops start at 0?

Show the step-by-step solution

The neutral value for addition is $0$ , so a sum that receives no terms remains $0$ . The neutral value for multiplication is $1$ , so a product that receives no terms remains $1$ :

\sum_{i\in\varnothing}a_i=0, \qquad \prod_{i\in\varnothing}a_i=1.

Starting a sum accumulator at 0 is correct. Starting a product accumulator at 0 is a bug: even a nonempty sequence becomes zero because every update has the form $0\times a_i=0$ . The product accumulator must start at 1.

Question 7

A loss array has shape (32, 5), organized as example × output. Which NumPy expression produces one mean for each example? Explain what the wrong axis would return.

Show the step-by-step solution

Each row contains the five output losses for one example. To combine the columns within every row, reduce axis 1:

per_example_loss = np.mean(losses, axis=1)

Axis 1 has size 5 and disappears, leaving shape (32,): one value for each of the 32 examples.

Using axis=0 would instead reduce the 32-example axis. It would leave shape (5,), containing one mean for each output position across the batch. Those numbers can be valid while answering the wrong question.

The bounds tell a sum where to start and stop

Product notation changes the starting value and operation

Four prediction errors become one mean squared error

NumPy mirrors the formula

An axis decides which indices disappear

Sum and mean carry different scale

Check your understanding

Sources

Follow new Math for AI articles

Keep reading

Logarithms for AI: Keep Tiny Probability Products from Vanishing

Powers for AI: Why Doubling Context Can Quadruple Attention Work

Tensor Indices for AI: Reading Samples, Tokens, and Features