Equations for AI: One Prediction Can Fit Many Parameter Pairs

A toy prediction model returns

$$\hat y=2w+b,$$

where the weight $w$ and bias $b$ are real numbers. For one observed example, the target is $5$. Matching that target imposes the constraint

$$2w+b=5.$$

The pair $(w,b)=(1,3)$ works because $2(1)+3=5$. So do $(0,5)$, $(2,1)$, and $(-1,7)$. One target has not revealed one privileged parameter setting. It has selected the full parameter-pair solution set

$$S=\left\{(w,b)\in\mathbb R^2\mid 2w+b=5\right\} =\left\{(t,5-2t)\mid t\in\mathbb R\right\}.$$

That set is a line in the $(w,b)$ plane: every point on it makes the model predict exactly $5$, and every point away from it misses the target. By the end of this lesson, you will be able to tell expressions, equations, constraints, and identities apart; describe all values that solve an equation; and recognize when an algebraic shortcut changes the solution set.

Learning outcome: Treat an equation as a constraint and describe all values satisfying it.

This lesson uses the domain discipline from Lesson 3: Domain, Codomain, and Valid Model Inputs: an equation’s solutions cannot be stated until its allowed values are known. The official Mathematics for Machine Learning companion organizes mathematical foundations as tools for understanding models and learning. Its Chapter 1 and Section 2.1 motivate the path from model parameters toward solution sets and later systems of linear equations. The single-equation bridge below is original; it stops before solving systems, Gaussian elimination, rank, or optimization.

An equals sign turns a calculation into a truth test

The symbols $2w+b$ form an expression. Once $w$ and $b$ have values, the expression has a numerical value. It does not claim that the value is any particular number.

The symbols $2w+b=5$ form an equation. They make a claim that can be true or false for a given assignment. With $(w,b)=(1,3)$ it is true; with $(1,2)$ it is false because $2(1)+2=4$.

These related terms do different jobs:

Term	Example over the real numbers	What it tells us
Expression	$2w+b$	A quantity to evaluate; it has no equals sign and no truth value
Equation	$2w+b=5$	A claim whose truth depends on $w$ and $b$
Constraint	Require $2w+b=5$	Only assignments satisfying the equation are feasible
Identity	$2(w+1)=2w+2$	True for every $w\in\mathbb R$
Solution	$(w,b)=(1,3)$	One assignment that makes the equation true
Solution set	$\{(t,5-2t)\mid t\in\mathbb R\}$	Every satisfying assignment, not just one example

The difference between “equation” and “constraint” is its role. The same string $2w+b=5$ is an equation as mathematical syntax and a constraint when it filters candidate parameters. “Identity” is stronger: $2(w+1)=2w+2$ does not filter any real $w$ at all.

One variable does not guarantee one answer

Consider the equation

$$3x-7=11, \qquad x\in\mathbb R.$$

Adding $7$ to both sides and then dividing both sides by the nonzero number $3$ gives

$$3x-7=11 \iff 3x=18 \iff x=6.$$

The double arrow $\iff$ means “if and only if.” Each step can be undone, so the real solution set is $S=\{6\}$. Substitution checks the answer in the original equation: $3(6)-7=18-7=11$.

The number of written variables does not determine the number of solutions:

Equation and stated domain	Solution set	Reason
$3x-7=11$, $x\in\mathbb R$	$\{6\}$	Exactly one real value passes the truth test
$x+1=x+2$, $x\in\mathbb R$	$\varnothing$	Subtracting $x+1$ gives the false statement $0=1$
$2(x+1)=2x+2$, $x\in\mathbb R$	$\mathbb R$	Both sides are equal for every real $x$
$x^2=2$, $x\in\mathbb R$	$\{-\sqrt2,\sqrt2\}$	Two real values square to $2$
$x^2=2$, $x\in\mathbb Q$	$\varnothing$	Neither real solution is rational

Here $\varnothing$ is the empty set, $\mathbb R$ is the set of real numbers, and $\mathbb Q$ is the set of rational numbers. The last two rows use the same equation but different domains, so they answer different questions.

One target leaves a parameter family

Return to $2w+b=5$ with $(w,b)\in\mathbb R^2$. The notation $\mathbb R^2$ means ordered pairs of real numbers. Isolate $b$:

$$2w+b=5 \iff b=5-2w.$$

Nothing in that step fixes $w$. Name its arbitrary real value $t$; then $b$ is forced to be $5-2t$. This produces every solution:

$$(w,b)=(t,5-2t), \qquad t\in\mathbb R.$$

Why is the description complete? For every real $t$, substitution gives $2t+(5-2t)=5$, so every listed pair is a solution. Conversely, any pair that satisfies $2w+b=5$ must satisfy $b=5-2w$; setting $t=w$ puts that pair in the listed family. The two directions establish equality of the sets.

Move along the exact solution set

Choose weight w

Arrow keys change w by 0.25; the required bias updates automatically.

1

Choose bias b

Arrow keys change b by 0.5; the required weight updates automatically.

3

Pointer option: drag the circle-with-cross marker along the line. The two sliders provide equivalent keyboard control.

Selected solution: w = 1, b = 3, and 2w + b = 5.

All exact solutions×Selected linked solution◆Independent candidate

Test any candidate pair

Set both values independently. The verdict uses exact quarter-step arithmetic, not a tolerance.

Candidate weight w

1

Candidate bias b

2

≠Not a solution: (1, 2) predicts 4, with residual -1. At this weight, the required bias is 3.

Selected solution: w = 1, b = 3, and 2w + b = 5. Not a solution: (1, 2) predicts 4, with residual -1. At this weight, the required bias is 3.

Current and reference parameter pairs, with calculations

Pair	Weight w	Bias b	2w + b	Residual	Constraint verdict
Selected linked pair	1	3	5	0	✓ Solution
Independent candidate	1	2	4	-1	≠ Not a solution
Reference pair A	0	5	5	0	✓ Solution
Reference pair B	2	1	5	0	✓ Solution

The solid line, selected circle-with-cross, candidate diamond, written verdict, and table carry the same conclusion without depending on color. Moving the linked controls changes the parameter pair but never the prediction: an increase of $0.25$ in $w$ requires a decrease of $0.5$ in $b$. Moving the candidate off the line produces a nonzero residual. One observation therefore establishes feasibility—it says which pairs fit—not which fitting pair should be preferred.

Reversible steps preserve every solution

Algebraic manipulation is safe when it produces an equivalent equation, not merely an equation that every old solution happens to satisfy. The distinction is the reverse direction.

Suppose $L(x)$ and $R(x)$ are defined for $x\in D$. For any expression $c(x)$ defined on that same domain,

$$L(x)=R(x) \iff L(x)+c(x)=R(x)+c(x).$$

The forward step adds the same value to both sides. The reverse step subtracts that same value, recovering the original equation. Because both implications hold for every $x\in D$, the two equations have exactly the same solution set.

Similarly, for a fixed nonzero number $k$,

$$L(x)=R(x) \iff kL(x)=kR(x).$$

Multiplication by $k$ is reversible because division by $k$ is defined. The condition $k\ne0$ is part of the argument, not a technical footnote.

For the one-variable example, the reversible path can be read in both directions:

$$3x-7=11 \xleftrightarrow[\text{subtract }7]{\text{add }7} 3x=18 \xleftrightarrow[\text{multiply by }3]{\text{divide by }3} x=6.$$

The labels below and above each arrow name inverse operations. A useful audit is simple: state the domain and ask how to undo every step for every candidate in that domain.

Four shortcuts can change the question

An operation may carry solutions forward without carrying all candidates back. Writing $\Rightarrow$ instead of $\iff$ keeps that one-way relationship honest.

Multiplying by zero erases information

Starting with $x=1$ and multiplying both sides by zero gives

$$x=1\Rightarrow 0x=0\cdot1\Rightarrow 0=0.$$

The original solution set is $\{1\}$; the final identity is true for every real $x$. There is no division by zero that can recover the original restriction.

Dividing by a variable can discard its zero case

Rewrite and factor $x^2=x$ as $x^2-x=0$ and then $x(x-1)=0$. The original real solutions are $x=0$ and $x=1$. Dividing the original equation by $x$ silently assumes $x\ne0$:

$$x^2=x \quad\xRightarrow[\text{assumes }x\ne0]{\text{divide by }x}\quad x=1.$$

The division did not prove that zero was invalid; it removed zero from the domain of the transformed step. Separate the $x=0$ case before dividing by an expression that could vanish.

Squaring can introduce an extra candidate

The equation $x=2$ has solution set $\{2\}$. Squaring both sides produces $x^2=4$, whose real solution set is $\{-2,2\}$. Both sides of the first equation being equal guarantees their squares are equal, but equal squares do not guarantee equal original signs. Any candidate created after squaring must be checked in the original equation.

A principal square root can hide a negative solution

From $x^2=4$, writing $x=\sqrt4=2$ loses $x=-2$. Over the real numbers,

$$\sqrt{x^2}=|x|,$$

not $x$ for every $x$. The complete step is $|x|=2$, hence $x=\pm2$. Squaring and taking a principal root are not inverse operations on all real numbers; sign and domain conditions determine which direction is valid.

NumPy tolerance is a numerical policy, not symbolic equality

In algebra, a pair belongs to $S$ only when $2w+b=5$ exactly. Software often evaluates decimal approximations, so an implementation may adopt a separate acceptance policy. NumPy’s official isclose documentation defines an element-wise tolerance test. With the target supplied as the reference and relative tolerance set to zero, this lesson asks whether

$$|\hat y-5|\le10^{-12}.$$

The following executed example makes the two questions visible:

import numpy as np

np.set_printoptions(precision=13)

candidates = np.array([
    [0.0, 5.0],
    [0.0, 5.0000000000005],
    [1.0, 2.999],
    [2.0, 1.0],
])
target = 5.0
predictions = 2.0 * candidates[:, 0] + candidates[:, 1]
residuals = predictions - target

print(predictions)
print(residuals)
print(predictions == target)
print(np.isclose(predictions, target, rtol=0.0, atol=1e-12))

[5.              5.0000000000005 4.999           5.             ]
[ 0.0000000000000e+00  5.0004445029117e-13 -9.9999999999945e-04
  0.0000000000000e+00]
[ True False False  True]
[ True  True False  True]

The first and fourth stored pairs produce a stored prediction equal to the stored target. The second does not, although its residual falls inside the chosen $10^{-12}$ engineering tolerance. The third misses by about $10^{-3}$ and fails both tests.

Neither Boolean mask proves an exact symbolic statement. == compares the stored floating-point values, while isclose applies a deliberately selected inequality. NumPy also documents that its general relative-tolerance formula treats the second argument as the reference and that the default absolute tolerance can be inappropriate for very small reference values. Naming the tolerances makes the implementation rule reviewable; substituting a candidate back into the original symbolic equation remains the mathematical check.

One observation tells us what fits, not what to choose

The parameter line exposes a practical limit before any training algorithm appears. The single observation “input coefficient $2$, target $5$” supplies one restriction on two real parameters. It can reject $(1,2)$, yet it cannot choose between $(1,3)$ and $(2,1)$ because both predictions are exactly $5$.

Additional observations can impose additional constraints, and later lessons will ask how constraints interact. This lesson’s stopping point is earlier and more fundamental: write the domain, treat the equation as a truth condition, describe the entire satisfying set, and keep every transformation accountable to an inverse or an explicit candidate check. That habit is the foundation for reading the next lesson’s derivations line by line.

Check your understanding

Question 1

Classify each item as an expression, conditional equation, identity, solution, or solution set: 3q + 1; 3q + 1 = 7; 2(q + 4) = 2q + 8; q = 2; and {2}. Use the domain q ∈ ℝ.

Show the step-by-step solution

• $3q+1$ is an expression: it has a value after $q$ is supplied but makes no equality claim.
• $3q+1=7$ is a conditional equation: it is true for some real values and false for others. Used to filter $q$, it is also a constraint.
• $2(q+4)=2q+8$ is an identity over $\mathbb R$. Expanding the left side gives $2q+8$ for every real $q$.
• $q=2$ states the solution after solving the conditional equation: $3(2)+1=7$.
• $\{2\}$ is the solution set. The braces describe the collection of every real solution, even though this collection has only one member.

The domain matters to the identity and solution-set statements; each classification above is made over $q\in\mathbb R$.

Question 2

Solve 5x + 2 = 17 over the real numbers, show reversible steps, and state the solution set.

Show the step-by-step solution

Subtract $2$ from both sides:

$$5x+2=17\iff5x=15.$$

This is reversible because adding $2$ recovers the first equation. Divide both sides by the nonzero number $5$:

$$5x=15\iff x=3.$$

Multiplying by $5$ reverses that step. Check the candidate in the original equation:

$$5(3)+2=15+2=17.$$

Therefore the real solution set is $S=\{3\}$.

Question 3

Describe every real pair (u, v) satisfying 3u + v = 7, then verify the pairs obtained at u = −1, 0, and 2.

Show the step-by-step solution

Isolate $v$ by subtracting $3u$ from both sides:

$$3u+v=7\iff v=7-3u.$$

Let $t$ stand for any real choice of $u$. The complete solution set is

$$S=\{(t,7-3t)\mid t\in\mathbb R\}.$$

Now substitute the three requested values:

$u$	$v=7-3u$	Check
$-1$	$10$	$3(-1)+10=-3+10=7$
$0$	$7$	$3(0)+7=7$
$2$	$1$	$3(2)+1=6+1=7$

Every row satisfies the equation, but the parameter $t$ ranges over all real numbers, so these three rows are examples rather than the whole set.

Question 4

A derivation changes 4(x − 3) = 20 into x − 3 = 5 and then x = 8. Explain why both steps preserve the real solution set, and check the result.

Show the step-by-step solution

First divide both sides by $4$:

$$4(x-3)=20\iff x-3=5.$$

Because $4\ne0$, multiplication by $4$ reverses this step. Next add $3$ to both sides:

$$x-3=5\iff x=8.$$

Subtracting $3$ reverses the second step. Both arrows therefore work in both directions for every real candidate. Substitution checks the result:

$$4(8-3)=4(5)=20.$$

The real solution set is $\{8\}$.

Question 5

Find every real solution of z² = 3z. Then explain which solution is lost if both sides are divided by z immediately.

Show the step-by-step solution

Move all terms to one side and factor:

$$z^2=3z \iff z^2-3z=0 \iff z(z-3)=0.$$

A product of real numbers is zero when at least one factor is zero, so

$$z=0\quad\text{or}\quad z-3=0,$$

which gives $z=0$ or $z=3$. Both check in the original equation: $0^2=3(0)=0$ and $3^2=3(3)=9$. Thus $S=\{0,3\}$.

Dividing $z^2=3z$ by $z$ assumes $z\ne0$ and leaves $z=3$. The step loses the valid solution $z=0$ because division by zero is undefined. The zero case must be separated before division.

Question 6

Solve √(x + 5) = x − 1 over the real numbers. Show how squaring creates a candidate that must be rejected.

Show the step-by-step solution

The square root requires $x+5\ge0$. Its value is nonnegative, so equality also requires $x-1\ge0$, or $x\ge1$. Square both sides to generate candidates:

$$x+5=(x-1)^2=x^2-2x+1.$$

Move all terms to one side and factor:

$$0=x^2-3x-4=(x-4)(x+1).$$

The squared equation gives $x=4$ or $x=-1$. The second candidate violates the required condition $x\ge1$ and also fails the original equation: $\sqrt{-1+5}=2$ while $-1-1=-2$. The first candidate checks:

$$\sqrt{4+5}=\sqrt9=3=4-1.$$

Therefore the original real solution set is $S=\{4\}$. Squaring preserved the true solution but introduced $-1$, so checking the original equation was essential.

Question 7

For 2w + b = 5, software computes a candidate prediction of 5 + 4 × 10⁻¹³. Is the pair an exact symbolic solution? Would np.isclose(prediction, 5, rtol=0, atol=10⁻¹²) accept it? Explain both answers.

Show the step-by-step solution

The exact residual stated in the prompt is

$$(5+4\times10^{-13})-5=4\times10^{-13},$$

which is not zero. Therefore the pair is not an exact solution of the symbolic equation as stated.

For the NumPy policy, compare the absolute residual with the absolute tolerance because rtol=0:

$$4\times10^{-13}\le10^{-12}.$$

That inequality is true, so np.isclose returns True. The result means “accepted under this tolerance,” not “proved symbolically equal.” If the application requires a tighter or scale-dependent policy, its tolerances must be chosen and documented accordingly.

Sources

1. Mathematics for Machine Learning companion website
2. Mathematics for Machine Learning book PDF
3. NumPy documentation: numpy.isclose