Reading AI Derivations: Catch the First Line That Breaks

Three model predictions produce residuals of $-2$ , $0$ , and $2$ . The signed residuals cancel, so their mean is zero. The middle prediction is exact, but the other two are wrong, and their mean squared error is $8/3$ . A plausible derivation can turn that real loss into zero by moving one square across one average.

The fastest way to debug the page of algebra is not to stare at its final answer. Start at the top and ask why each line follows from the line before it. Stop at the first broken equals sign. Every later line may be flawless arithmetic applied to an expression that is no longer the one the derivation claimed to calculate.

By the end of this lesson, you will be able to read a derivation line by line, name the rule behind each transformation, and expose an unstated assumption about a domain, divisor, sign, or index range. You will also debug the loss calculation in algebra and NumPy without using calculus or optimization.

The lesson combines two earlier skills. Lesson 6 on sums and loss aggregation established how bounds and averaging turn indexed losses into one number. Lesson 9 on equations and solution sets showed why an algebraic move must preserve the mathematical claim being made. The official Mathematics for Machine Learning companion and its introductory chapter motivate mathematical foundations as tools for understanding models, debugging methods, and making their assumptions visible. The audit method and examples below are original.

Every line makes a local claim

A derivation is a sequence of mathematical statements in which each new line is supported by a definition, an algebraic rule, a stated assumption, or a result already established. Reading it well is a local task. Between two adjacent lines, ask four questions:

What changed: a symbol, an operation, a bound, or an entire expression?
Which definition or algebraic rule permits that exact change?
Which conditions make the rule legal?
Does the new line make the same claim, or only a related one?

The symbol connecting expressions tells you what kind of claim to audit.

Symbol	Read it as	Its job in a derivation
$:=$	“is defined as”	Introduces a name or convention; the right side specifies what the left side means
$=$	“has the same value as”	Claims equality under the current domain and assumptions
$\iff$	“if and only if”	Claims two conditions have exactly the same solutions
$\Rightarrow$	“implies”	Preserves only one direction; reversing it needs a separate reason
$\approx$	“is approximately equal to”	Reports a numerical approximation, not an exact identity

This method deliberately separates two questions. “Did the author reach the expected answer?” is a test of the destination. “Did every step preserve the claim?” is a test of the route. A correct destination can still be reached by invalid reasoning, just as a false destination can follow many valid steps after one early substitution error.

One moved square erases a real loss

For $N$ examples, let $\hat y_i$ be prediction $i$ , $y_i$ its target, and $r_i$ its residual. We define

r_i:=\hat y_i-y_i

and define mean squared error (MSE) by

L_{\mathrm{MSE}}:=\frac{1}{N}\sum_{i=1}^{N}r_i^2, \qquad N>0.

The condition $N>0$ is not decoration: an arithmetic mean divides by the number of terms, so an empty batch has no MSE under this definition.

Now use three residuals,

(r_1,r_2,r_3)=(-2,0,2).

Open the steps in this derivation map. The status words and symbols repeat the color cue, and every summary works with the keyboard through the browser’s native disclosure control.

Audit a loss derivation from its definition to its claimed zero value.

Open any line to inspect the rule and its assumptions. Line 4 contains the first broken equals sign.

ValidLine 1L_MSE := (1/3) Σᵢ₌₁³ rᵢ²
Why this move works—or fails
This line instantiates the definition of MSE with N = 3.
Condition to check
The batch contains exactly three residuals, so N = 3 > 0.
ValidLine 2= (1/3)((−2)² + 0² + 2²)
Why this move works—or fails
Substitute the three residual values and expand the finite sum.
Condition to check
The index range remains i = 1, 2, 3 in the same order.
ValidLine 3= (1/3)(4 + 0 + 4) = 8/3
Why this move works—or fails
Square each residual, add the three squared values, then divide by 3.
Condition to check
Ordinary real-number arithmetic applies.
InvalidLine 4= ((1/3) Σᵢ₌₁³ rᵢ)²
Why this move works—or fails
A square cannot generally move outside a sum or mean. The new expression is squared mean residual, not MSE.
Condition to check
Equality would need a special condition on these residuals; none was stated, and it is false here.
Valid for the new expressionLine 5= ((−2 + 0 + 2)/3)²
Why this move works—or fails
This correctly expands the mean residual introduced on Line 4.
Condition to check
The three residuals and denominator are unchanged.
Invalid conclusion about MSELine 6= 0² = 0, therefore L_MSE = 0
Why this move works—or fails
Zero is correct for squared mean residual, but Line 4 severed the connection to MSE.
Condition to check
The conclusion silently assumes the invalid equality on Line 4.

The first three lines compute the actual loss:

L_{\mathrm{MSE}} =\frac{(-2)^2+0^2+2^2}{3} =\frac{8}{3} \approx2.6667.

Line 4 replaces it with a different quantity:

L_{\mathrm{SMR}} :=\left(\frac{1}{3}\sum_{i=1}^{3}r_i\right)^2 =\left(\frac{-2+0+2}{3}\right)^2 =0.

Here $L_{\mathrm{SMR}}$ is a name for squared mean residual. Opposite errors cancel before the square is applied. MSE squares first, so both $-2$ and $2$ contribute $4$ . The order of operations changes what the number measures.

Example $i$	Residual $r_i$	Contribution to $\sum r_i$	Contribution to $\sum r_i^2$
1	$-2$	$-2$	$4$
2	$0$	$0$	$0$
3	$2$	$2$	$4$
Total		$0$	$8$

The illegal move is also visible by expanding a square of a sum:

(r_1+r_2+r_3)^2 =r_1^2+r_2^2+r_3^2 +2r_1r_2+2r_1r_3+2r_2r_3.

The cross terms do not vanish automatically, and the outer mean contributes a factor of $1/9$ , not $1/3$ . Moving the square therefore changes both the terms and their scale.

Rules travel with their conditions

Many derivations hide their most important information in a phrase such as “for $x\ne0$ ” or “over the real numbers.” Remove the condition and a legal identity can become a false universal claim.

Consider cancellation:

\frac{x}{x}=1 \qquad \text{only when }x\ne0.

At $x=0$ , the left side is undefined. Likewise, dividing the equation $x(x+1)=x$ by $x$ produces $x+1=1$ , but only under the added assumption $x\ne0$ . A quick glance may suggest that $x=0$ and $x=-1$ are solutions. The line audit rejects that guess as soon as the original equation is simplified:

x(x+1)=x \iff x^2=0 \iff x=0.

The tempting division by $x$ happens to return $x=0$ from $x+1=1$ , even though that very step assumed $x\ne0$ . Its result contradicts its own license. The safe route rearranges the equation without dividing by a possibly zero expression.

Squaring also needs a direction check. If $a=b$ , then $a^2=b^2$ . The reverse does not follow without more information: $a^2=b^2$ allows $a=b$ or $a=-b$ . Squaring both sides of an equation can therefore add candidates that must be checked in the original equation.

Square roots carry a sign condition too. For real $x$ ,

\sqrt{x^2}=|x|,

not always $x$ . Substituting $x=-3$ gives $\sqrt{(-3)^2}=3$ , whereas the unstated replacement $\sqrt{x^2}=x$ would give $-3$ . Consequently, $x^2=9$ has two real solutions, $x=3$ and $x=-3$ ; writing only $x=\sqrt9$ loses the negative solution.

These conditions do not make derivations fussy. They identify which inputs the argument actually covers.

Bounds and definitions must stay fixed

An indexed derivation has another kind of hidden state: its range. These two expressions do not usually match:

\sum_{i=1}^{N}r_i^2 \qquad\text{and}\qquad \sum_{i=1}^{N-1}r_i^2.

The second omits example $N$ . Renaming a dummy index is harmless,

\sum_{i=1}^{N}r_i^2=\sum_{j=1}^{N}r_j^2,

because the bounds and term pattern stay the same. Changing a bound, the denominator, or the operation is a mathematical change that needs a reason.

Definitions need the same discipline. Once $r_i:=\hat y_i-y_i$ , a later line cannot quietly use $r_i:=y_i-\hat y_i$ . Squaring hides that particular sign reversal because $r_i^2=(-r_i)^2$ , but an unsquared mean would change sign. A derivation that works only because a later operation masks an inconsistent definition is still broken.

NumPy makes the order visible

NumPy documents np.square as an element-wise operation: each residual is squared while the input shape is preserved. It documents np.mean as an arithmetic reduction over the selected axis, or over the flattened array by default. Their nesting therefore encodes the derivation’s order.

import numpy as np

predictions = np.array([8.0, 10.0, 12.0])
targets = np.array([10.0, 10.0, 10.0])
residuals = predictions - targets

mse = np.mean(np.square(residuals))
squared_mean_residual = np.square(np.mean(residuals))

print(residuals)
print(mse)
print(squared_mean_residual)

[-2.  0.  2.]
2.6666666666666665
0.0

np.mean(np.square(residuals)) says “square each value, then average.” np.square(np.mean(residuals)) says “average first, then square the one result.” Parentheses are a compact derivation tree; swapping the calls changes the quantity.

A numerical test has asymmetric logical power. The counterexample above is enough to falsify the universal claim

\frac{1}{N}\sum_{i=1}^{N}r_i^2 =\left(\frac{1}{N}\sum_{i=1}^{N}r_i\right)^2 \quad\text{for every residual sequence}.

One failure refutes “for every.” One success does not prove it. With residuals $(2,2,2)$ , both expressions equal $4$ , but that passing case cannot establish the identity for all other sequences. A general proof needs algebra that covers the stated domain; tests are excellent at finding counterexamples and implementation mistakes.

Carry the audit into AI code and papers

Loss equations in model code often compress several moves: define residuals, square element-wise, reduce over an example axis, and divide by a count. The line audit makes each move inspectable. Write the intended formula, match one code operation to each mathematical operation, keep the same examples in the same reduction, and state what happens when the batch is empty.

The same habit scales to longer derivations without requiring more advanced mathematics. Keep a margin for the rule and its conditions. Mark := when a quantity receives a name. Use $=$ only when two expressions have the same value under the active assumptions. Use $\iff$ only when both directions preserve the same solution set. When the route breaks, report the first bad transition—not merely a suspicious final number.

That practice is the bridge from prerequisite algebra to the course’s next module. Dense notation becomes manageable when every line must answer one small question: what changed, and why was that change allowed?

Check your understanding

Question 1

Classify each connector: a := b + 1; x/x = 1 for x ≠ 0; x² = 9 ⇔ x = 3; and 1/3 ≈ 0.333. Explain what each symbol claims.

Show the step-by-step solution

In $a:=b+1$ , $:=$ introduces a definition: the expression $b+1$ specifies what the new name $a$ means.
In $x/x=1$ for $x\ne0$ , $=$ states a conditional equality. It is valid on the stated nonzero domain and undefined at $x=0$ .
The claim $x^2=9\iff x=3$ is false over the reals. The left condition also has $x=-3$ , so the two sides do not have the same solution set. A valid replacement is $x^2=9\iff x\in\{-3,3\}$ .
In $1/3\approx0.333$ , $\approx$ marks an approximation. The decimal shown is close to, but not exactly equal to, one third.

Each connector makes a different kind of promise; reading them all as ordinary equality would discard essential information.

Question 2

For residuals (−1, 2, 2), compute mean squared error and squared mean residual. Identify the first operation at which their calculations differ.

Show the step-by-step solution

MSE squares each residual first:

L_{\mathrm{MSE}} =\frac{(-1)^2+2^2+2^2}{3} =\frac{1+4+4}{3} =3.

Squared mean residual averages first:

L_{\mathrm{SMR}} =\left(\frac{-1+2+2}{3}\right)^2 =1^2 =1.

The first difference is the operation immediately applied to the residuals: MSE applies the square element by element, while squared mean residual applies the sum and division before the square. The values $3$ and $1$ confirm that the two operation orders are not interchangeable.

Question 3

Audit this derivation for N = 3: Σᵢ₌₁³ rᵢ² = r₁² + r₂² + r₃² = r₁² + r₂². Which equals sign breaks first, and what unstated assumption would make it valid?

Show the step-by-step solution

The first equality is valid because it expands the inclusive index range $i=1,2,3$ :

\sum_{i=1}^{3}r_i^2=r_1^2+r_2^2+r_3^2.

The second equality drops $r_3^2$ , so it is the first broken equals sign. It would be valid only under an added condition that makes the omitted term zero:

r_3^2=0\iff r_3=0.

Without that condition, the derivation has silently changed the fixed index range from three examples to two effective terms.

Question 4

Solve x(x + 1) = x over the real numbers. Explain why dividing both sides by x is not a valid route to the solution, even though the simplified equation appears to produce x = 0.

Show the step-by-step solution

First subtract $x$ from both sides and simplify:

x(x+1)=x \iff x^2+x-x=0 \iff x^2=0 \iff x=0.

Substitution checks the solution: $0(0+1)=0$ .

Dividing the original equation by $x$ requires $x\ne0$ and gives $x+1=1$ , hence $x=0$ . That result violates the nonzero condition required by the division. The route has no valid candidate in its assumed domain, even though its symbols end at the original solution. The safe derivation never divides by the possibly zero expression.

Question 5

A derivation over the real numbers changes √(z²) = 5 into z = 5. Find the missing case and name the invalid replacement.

Show the step-by-step solution

For real $z$ , the principal square root is nonnegative and

\sqrt{z^2}=|z|.

The equation is therefore

|z|=5,

which has $z=5$ and $z=-5$ . Both check because $\sqrt{5^2}=5$ and $\sqrt{(-5)^2}=5$ . The derivation incorrectly replaced $\sqrt{z^2}$ with $z$ without assuming $z\ge0$ ; that loses the negative case.

Question 6

A student tests residuals (2, 2, 2) and finds that MSE equals squared mean residual. Does this prove the formulas are identical? Give a counterexample and explain the logic.

Show the step-by-step solution

For $(2,2,2)$ , both quantities are indeed $4$ :

\frac{2^2+2^2+2^2}{3}=4, \qquad \left(\frac{2+2+2}{3}\right)^2=4.

That establishes equality for one input sequence only. It does not cover every allowed sequence. Use $(-2,0,2)$ as a counterexample:

\frac{(-2)^2+0^2+2^2}{3}=\frac83, \qquad \left(\frac{-2+0+2}{3}\right)^2=0.

Because a universal identity claims equality for every sequence, one counterexample falsifies it. No finite collection of passing examples alone proves the universal algebraic claim.

Question 7

Debug this NumPy line for mean squared error: loss = np.square(np.mean(predictions - targets)). Write the corrected line and map its operations to the formula.

Show the step-by-step solution

The original line computes squared mean residual because np.mean runs before np.square. The corrected MSE is

residuals = predictions - targets
loss = np.mean(np.square(residuals))

The first line implements $r_i:=\hat y_i-y_i$ . The inner call np.square(residuals) implements the element-wise terms $r_i^2$ . The outer np.mean(...) adds those squared terms and divides by the number of included elements. The code order now matches

L_{\mathrm{MSE}}:=\frac1N\sum_{i=1}^{N}r_i^2, \qquad N>0.

The nonempty-input condition is still part of the implementation contract.

Question 8

A derivation renames Σᵢ₌₁ᴺ aᵢ as Σⱼ₌₁ᴺ aⱼ, then replaces it with Σⱼ₌₀ᴺ aⱼ. Audit both steps.

Show the step-by-step solution

The first step is valid:

\sum_{i=1}^{N}a_i=\sum_{j=1}^{N}a_j.

The index letter is a local placeholder. Renaming $i$ to $j$ changes neither the bounds nor the term selected at each position.

The second step changes the lower bound from $1$ to $0$ :

\sum_{j=0}^{N}a_j =a_0+\sum_{j=1}^{N}a_j.

It equals the original sum only if $a_0=0$ is defined and explicitly assumed. Otherwise the new sum contains one extra term—or may be undefined if the sequence has no index $0$ . The first broken equals sign is therefore the one that changes the lower bound.

Every line makes a local claim

One moved square erases a real loss

Rules travel with their conditions

Bounds and definitions must stay fixed

NumPy makes the order visible

Carry the audit into AI code and papers

Check your understanding

Sources

Follow new Math for AI articles

Keep reading

Equations for AI: One Prediction Can Fit Many Parameter Pairs

Logarithms for AI: Keep Tiny Probability Products from Vanishing

Powers for AI: Why Doubling Context Can Quadruple Attention Work