Two Observations, One Parameter Pair: Systems of Equations for AI

A tiny prediction rule has two unknown parameters:

$$\widehat y=wx+b.$$

The first observation says that input $x=1$ produced target $y=5$. That does not reveal the weight $w$ or bias $b$ separately. It gives one constraint, $w+b=5$, whose parameter pairs form an entire line. A second observation, $(x,y)=(3,9)$, gives another line, $3w+b=9$.

Only one point lies on both lines: $(w,b)=(2,3)$. At that point, the same model fits both rows. This shared point is the central idea of a system of linear equations: each equation imposes a constraint, and a solution must pass every constraint at the same time.

By the end of this lesson, you will be able to interpret a system as simultaneous constraints, solve a two-equation system by substitution or subtraction, classify two lines as intersecting, parallel, or coincident, and recover two exact model parameters from two observations.

Learning outcome: Interpret a system as simultaneous constraints and classify its intersections.

This lesson continues Lesson 11: Linear Equations for AI, where one equation became one geometric line and substitution tested whether a point belonged to it. The official Mathematics for Machine Learning companion places systems of linear equations at the opening of linear algebra. Its Section 2.1 connects simultaneous solutions with geometric intersections. The parameter-recovery scenario, derivations, code, visual, and assessments below are original.

One solution must pass two truth tests

Consider two linear equations in the same real unknowns $w$ and $b$:

$$\begin{aligned} a_1w+b&=c_1,\\ a_2w+b&=c_2. \end{aligned}$$

The fixed real numbers $a_1$ and $a_2$ multiply the weight. The fixed real numbers $c_1$ and $c_2$ are the required outputs. The ordered pair $(w,b)$ is the unknown parameter setting.

The word “simultaneously” prevents a common mistake. The pair $(0,5)$ solves $w+b=5$, but it fails the second equation because $3(0)+5=5\ne9$. The pair $(3,0)$ solves $3w+b=9$, but fails the first because $3+0\ne5$. Each pair fits one row; neither fits the system.

Candidate $(w,b)$	Test $w+b=5$	Test $3w+b=9$	Solves the system?
$(0,5)$	$0+5=5$	$0+5=5$	No
$(3,0)$	$3+0=3$	$9+0=9$	No
$(2,3)$	$2+3=5$	$6+3=9$	Yes

A system is not solved until the same assignment passes every original truth test.

Subtraction isolates what the observations changed

The opening system is

\begin{aligned} w+b&=5, \tag{1}\\ 3w+b&=9. \tag{2} \end{aligned}

The two equations contain the same $+b$. Subtracting the entire left side of equation (1) from equation (2), and doing the same on the right, removes $b$:

$$\begin{aligned} (3w+b)-(w+b)&=9-5,\\ 3w+b-w-b&=4,\\ 2w&=4,\\ w&=2. \end{aligned}$$

Now substitute $w=2$ into either original equation. Using equation (1),

$$2+b=5 \iff b=3.$$

The subtraction has an AI interpretation. Between the two observations, the input rises by $3-1=2$ and the output rises by $9-5=4$. The bias contributes the same amount to both predictions, so it disappears from the difference. The weight is the output change per unit of input change:

$$w=\frac{9-5}{3-1}=\frac42=2.$$

Once the slope-like weight is known, either row reveals the bias. This is an exact two-point fit, not a description of ordinary large-dataset training. Real observations can include noise, and a third row may disagree with the line determined by the first two. Later lessons will develop tools for those cases.

The algebraic answer is the geometric crossing

Each observation can be rearranged as a line in the $(w,b)$ parameter plane:

$$\begin{aligned} w+b=5 &\iff b=5-w,\\ 3w+b=9 &\iff b=9-3w. \end{aligned}$$

This graph is not the usual plot of observed input $x$ against target $y$. Its horizontal coordinate is a candidate weight and its vertical coordinate is a candidate bias. Every point on the first line fits observation 1. Every point on the second line fits observation 2. Their crossing at $(2,3)$ fits both.

Change the second observation below. The solid line remains $w+b=5$; the dashed line becomes $x_2w+b=y_2$. Native sliders expose the second input $x_2$ and target $y_2$, while the three buttons jump to representative intersection states.

Change the second observation

Second input x₂

Sets the weight coefficient in x₂w + b = y₂. The first input stays fixed at 1.

3

Second target y₂

Sets the right side of the second constraint. The first target stays fixed at 5.

9

Explore a representative configuration

One intersectionParallel linesSame line

One solution: the lines meet at weight w = 2 and bias b = 3.

One solution: the lines meet at weight w = 2 and bias b = 3.

Observation 1: w + b = 5 (solid)Observation 2: x₂w + b = y₂ (dashed)⊗Shared solution (circled plus)

Current constraints and text alternative for the graph

Quantity	Current value	Interpretation
Observation 1 constraint	w + b = 5	Every point on the solid line fits the first observation.
Observation 2 constraint	3w + b = 9	Every point on the dashed line fits the second observation.
Intersection class	One shared point	One solution: the lines meet at weight w = 2 and bias b = 3.
Recovered weight w	2	This coordinate belongs to the shared solution.
Recovered bias b	3	This coordinate belongs to the shared solution.

The component’s first server-rendered state is the worked system: $x_2=3$, $y_2=9$, and $(w,b)=(2,3)$. Its status sentence and table contain the conclusion independently of the SVG and remain useful without JavaScript. The line labels also use solid and dashed patterns, while the intersection has its own circled-plus shape, so color is not the only state cue.

Three pictures exhaust the two-line possibilities

For the family

$$\begin{aligned} a_1w+b&=c_1,\\ a_2w+b&=c_2, \end{aligned}$$

subtracting the first equation from the second gives

$$(a_2-a_1)w=c_2-c_1.$$

The two differences classify the geometry without requiring matrices.

Coefficient comparison	Target comparison	Geometry	System solution set
$a_2\ne a_1$	Any values	Intersecting	One point
$a_2=a_1$	$c_2\ne c_1$	Parallel	Empty: $\varnothing$
$a_2=a_1$	$c_2=c_1$	Coincident	The entire shared line

When $a_2\ne a_1$, division is legal and produces one weight:

$$w=\frac{c_2-c_1}{a_2-a_1}, \qquad b=c_1-a_1w.$$

When $a_2=a_1$, the left-side difference is zero. If the targets differ, the subtracted equation becomes $0=c_2-c_1$ with a nonzero right side, a contradiction. If the targets also match, subtraction becomes the identity $0=0$: the second equation repeats the first and contributes no new restriction.

The next lesson will turn this preview into a systematic study of no, one, and many solutions. For now, the geometric test is enough: crossing lines share one point, distinct parallel lines share none, and coincident lines share all their points.

NumPy verifies the fit and Matplotlib plots the constraints

The calculation can be implemented without hiding it inside a general solver. The function below accepts exactly two observations, handles the repeated-input boundary, and checks any recovered parameters against both original rows.

import numpy as np
import matplotlib.pyplot as plt


def recover_two_parameter_model(x, y, atol=1e-12):
    delta_x = x[1] - x[0]
    delta_y = y[1] - y[0]

    if np.isclose(delta_x, 0.0, rtol=0.0, atol=atol):
        if np.isclose(delta_y, 0.0, rtol=0.0, atol=atol):
            return "many", None
        return "none", None

    weight = delta_y / delta_x
    bias = y[0] - weight * x[0]
    residual = weight * x + bias - y
    return "one", (weight, bias, residual)


x = np.array([1.0, 3.0])
y = np.array([5.0, 9.0])
classification, result = recover_two_parameter_model(x, y)
weight, bias, residual = result

print("classification:", classification)
print("weight:", weight)
print("bias:", bias)
print("predictions:", weight * x + bias)
print("residual:", residual)
print(
    "both constraints satisfied:",
    np.isclose(residual, 0.0, rtol=0.0, atol=1e-12),
)

candidate_weight = np.linspace(-1.0, 5.0, num=121)
bias_from_first = y[0] - x[0] * candidate_weight
bias_from_second = y[1] - x[1] * candidate_weight

plt.plot(candidate_weight, bias_from_first, label="w + b = 5")
plt.plot(
    candidate_weight,
    bias_from_second,
    "--",
    label="3w + b = 9",
)
plt.plot([weight], [bias], "o", label="shared solution")
plt.xlabel("weight w")
plt.ylabel("bias b")
plt.legend()
plt.show()

The executed output for the intersecting case is:

classification: one
weight: 2.0
bias: 3.0
predictions: [5. 9.]
residual: [0. 0.]
both constraints satisfied: [ True  True]

numpy.isclose compares numbers using explicit relative and absolute tolerances. Here the reference value is zero, so rtol=0.0 and atol=1e-12 make the intended absolute threshold clear. That test checks this floating-point execution; it does not replace the exact algebraic derivation.

numpy.linspace returns evenly spaced samples over the chosen interval. The 121 sampled weights give Matplotlib enough line nodes for a clean plot, but the samples are not the mathematical solution set. Each line contains infinitely many real points. The Matplotlib plot documentation specifies that the supplied x and y values provide the coordinates of line nodes; calling it twice draws the two constraint lines.

Exact fit can fail when the dataset asks for more

The recovered model $\widehat y=2x+3$ fits $(1,5)$ and $(3,9)$. Suppose a third observation is $(2,8)$. The model predicts

$$\widehat y=2(2)+3=7,$$

so its residual is $7-8=-1$. In the parameter plane, the new row contributes a third line, $2w+b=8$, which does not pass through $(2,3)$. There is no single parameter pair that fits all three rows exactly.

This does not make the first calculation wrong. It changes the problem. Two rows selected the exact line through those two observations; the third row showed that the full dataset does not lie on it. Machine-learning practice often replaces “satisfy every equation exactly” with “find parameters whose predictions are collectively close,” but that approximation problem belongs to later lessons.

Another misconception runs in the opposite direction. If two observations are identical, such as $(1,5)$ and $(1,5)$, the system has not learned the two parameters. It has written $w+b=5$ twice. Every point on that one line still fits, so the weight and bias remain underdetermined.

The dataset claim lives at the intersection

One observation can certify that a parameter pair fits one row. A system raises the standard: the same pair must survive every row’s test. For the two-row example, subtracting observations exposes the weight, substitution recovers the bias, and the crossing of the two parameter lines shows why the answer is shared.

That geometric habit will carry into the next lesson. When a new constraint arrives, inspect what it adds: a new crossing, a contradiction, or merely the same line drawn again.

Check your understanding

Question 1

Test (w, b) = (1, 4), (2, 3), and (3, 0) against the system w + b = 5 and 3w + b = 9. Which pair, if any, solves the system?

Show the step-by-step solution

Substitute each pair into both original equations.

For $(1,4)$,

$$1+4=5, \qquad 3(1)+4=7\ne9.$$

It fits the first constraint only. For $(2,3)$,

$$2+3=5, \qquad 3(2)+3=9.$$

It fits both constraints. For $(3,0)$,

$$3+0=3\ne5, \qquad 3(3)+0=9.$$

It fits the second constraint only. Therefore $(2,3)$ is the sole listed pair that solves the system. Passing either truth test alone is insufficient.

Question 2

Solve 2u + v = 8 and 5u + v = 17 by subtraction. Verify the result in both equations and identify the intersection point.

Show the step-by-step solution

Subtract the first equation from the second. Parentheses keep the subtraction attached to every term:

$$\begin{aligned} (5u+v)-(2u+v)&=17-8,\\ 3u&=9,\\ u&=3. \end{aligned}$$

Substitute $u=3$ into the first equation:

$$2(3)+v=8 \iff 6+v=8 \iff v=2.$$

Now verify both originals:

$$2(3)+2=8, \qquad 5(3)+2=17.$$

The solution and geometric intersection point are $(u,v)=(3,2)$.

Question 3

An affine model ŷ = wx + b observes (x, y) = (−1, 1) and (2, 10). Recover w and b, then write the fitted prediction rule.

Show the step-by-step solution

Translate the observations into parameter constraints:

$$\begin{aligned} -w+b&=1,\\ 2w+b&=10. \end{aligned}$$

Subtract the first equation from the second:

$$(2w+b)-(-w+b)=10-1 \iff 3w=9 \iff w=3.$$

Substitute $w=3$ into $-w+b=1$:

$$-3+b=1 \iff b=4.$$

The fitted rule is $\widehat y=3x+4$. It returns $3(-1)+4=1$ and $3(2)+4=10$, so both residuals are zero.

Question 4

Classify the intersection of 2x + y = 4 and 2x + y = 7. Show how subtraction proves the classification.

Show the step-by-step solution

Subtract the first equation from the second:

$$(2x+y)-(2x+y)=7-4 \iff 0=3.$$

The statement $0=3$ is false for every $(x,y)$. Both lines have the same left side, so they have the same slope, but their different right sides place them at different intercepts. They are distinct parallel lines. Their intersection, and therefore the system’s solution set, is $\varnothing$.

Question 5

Classify the system 2x + 2y = 10 and x + y = 5. Explain why writing two equations does not create a unique solution.

Show the step-by-step solution

Divide the first equation by the nonzero number $2$:

$$2x+2y=10 \iff x+y=5.$$

The result is exactly the second equation. Both equations therefore draw the same line. Every point $(t,5-t)$ with $t\in\mathbb R$ satisfies both:

$$t+(5-t)=5$$

and doubling that equality gives $2t+2(5-t)=10$. The system has infinitely many solutions along one coincident line; the second equation repeats rather than narrows the first constraint.

Question 6

A student subtracts w + b = 5 from 3w + b = 9 and writes 2w + 2b = 4. Find the first error, finish the solution correctly, and verify it.

Show the step-by-step solution

Subtraction must apply to the entire first left side:

$$(3w+b)-(w+b)=3w+b-w-b.$$

The student’s $+2b$ is a sign-distribution error. Because $b-b=0$, the correct step is

$$2w=4 \iff w=2.$$

Substituting into $w+b=5$ gives $2+b=5$, so $b=3$. Verification in both original equations yields

$$2+3=5, \qquad 3(2)+3=9.$$

Thus $(w,b)=(2,3)$ is the solution.

Question 7

The model ŷ = 2x + 3 fits observations (1, 5) and (3, 9). A third observation is (2, 8). Does the same parameter pair solve all three constraints? Compute and interpret the residual.

Show the step-by-step solution

Evaluate the model at the third input:

$$\widehat y=2(2)+3=7.$$

Using residual $r=\widehat y-y$, the third residual is

$$r=7-8=-1.$$

The pair $(w,b)=(2,3)$ still solves the first two constraints, but it fails the third because $2w+b=2(2)+3=7\ne8$. Therefore no, this parameter pair does not solve the three-equation system. The negative residual means its prediction is one unit below the third target.

Sources

1. Mathematics for Machine Learning companion website
2. Mathematics for Machine Learning book PDF
3. NumPy documentation: numpy.linspace
4. NumPy documentation: numpy.isclose
5. Matplotlib documentation: matplotlib.pyplot.plot