No solution, one solution, or many solutions

Three two-row datasets can make the same model behave in three radically different ways. Let the affine prediction rule be

$$\widehat y=wx+b,$$

where the real numbers $w$ and $b$ are the unknown weight and bias.

Dataset	First observation	Second observation	Exact-fit result
A	$(1,4)$	$(3,8)$	One pair: $(w,b)=(2,2)$
B	$(1,4)$	$(1,6)$	No pair: the same input demands two different outputs
C	$(1,4)$	$(1,4)$	Infinitely many pairs: the second row repeats the first

Dataset A identifies both parameters. Dataset B contradicts the exact model. Dataset C adds a row but no new restriction. The number of rows alone cannot tell us how much information the data contain.

By the end of this lesson, you will be able to recognize inconsistent, unique, and underdetermined two-variable linear systems from equations and geometry; justify each classification without determinants or matrix rank; and explain what the result says about parameter identifiability.

Learning outcome: Recognize inconsistent, unique, and underdetermined systems.

This lesson builds on Lesson 12: Systems of Equations as Intersections, where one solution had to satisfy every constraint and appeared as a shared point. The official Mathematics for Machine Learning companion develops the no, one, or infinitely many classification in Sections 2.1 and 2.3. The datasets, derivations, code, explorer, and assessments here are original.

A solution count diagnoses the available information

Consider two equations in the real unknowns $u$ and $v$. Each nondegenerate equation—one whose two left-side coefficients are not both zero—draws a line in the $(u,v)$ plane. The system’s solution set is the intersection of those lines.

The three possibilities are not a memorized catalog. They follow from the geometry of lines. Two nonparallel lines meet once. Distinct parallel lines never meet. If two lines share two different points, both pass through the one line determined by those points, so they coincide and share every point. Two linear equations in two variables therefore cannot have exactly two, three, or any other finite number greater than one of real solutions.

In machine-learning language, parameters are identifiable from the given observations when only one parameter setting fits them under the stated model. Identifiability depends on the information supplied by the constraints, not simply on how many rows were collected.

Elimination reveals a value, a contradiction, or an identity

The most direct classification test at this stage of the course is to combine the equations while preserving their shared solutions. Watch what the last line of the calculation says.

One solution leaves a value to substitute

Solve

\begin{aligned} 2u+v&=7, \tag{1}\\ u-v&=2. \tag{2} \end{aligned}

Adding equations (1) and (2) removes $v$:

$$3u=9 \iff u=3.$$

Substitute $u=3$ into equation (2):

$$3-v=2 \iff v=1.$$

No solution ends in a contradiction

Now consider

\begin{aligned} u+2v&=4, \tag{3}\\ 2u+4v&=11. \tag{4} \end{aligned}

Doubling equation (3) gives $2u+4v=8$. Equation (4) demands that the identical left side equal $11$. Subtracting twice equation (3) from equation (4) makes the conflict explicit:

$$(2u+4v)-2(u+2v)=11-2(4) \iff 0=3.$$

No values of $u$ and $v$ can make $0=3$ true. The system is inconsistent, its two lines are parallel and distinct, and its solution set is $\varnothing$. The calculation did not fail to find a hidden answer; it proved that no exact answer exists.

Infinitely many solutions end in an identity

Finally, take

\begin{aligned} 3u-v&=5, \tag{5}\\ -6u+2v&=-10. \tag{6} \end{aligned}

Equation (6) is $-2$ times equation (5). Adding twice equation (5) to equation (6) gives

$$(-6u+2v)+2(3u-v)=-10+2(5) \iff 0=0.$$

The identity $0=0$ is true for every $u$ and $v$; it contributes no new restriction. It does not mean every pair solves the original system. One independent equation still remains. Choose any real number $t$ for $u$, then equation (5) forces $v=3t-5$:

$$(u,v)=(t,3t-5), \qquad t\in\mathbb R.$$

For every real $t$,

$$3t-(3t-5)=5,$$

and multiplying this equality by $-2$ verifies equation (6). The free choice of $t$ generates infinitely many solutions along the shared line.

Compare all three geometries in one parameter plane

The explorer keeps $w+b=4$ as one observation constraint, then changes what the remaining data demand. Choose a configuration with the native select. The lines transition between states unless the device requests reduced motion. Solid, dashed, and dotted patterns carry the same distinctions as color, and the captioned table states every conclusion without relying on the SVG or JavaScript.

Choose an observation configuration

Use the arrow keys to move through the options. The graph, status, and table update to the selected exact-fit problem.

One solutionNo solutionInfinitely many solutionsThree nearly agreeing observations

The lines cross once at (w, b) = (2, 2), so both parameters are identifiable.

Constraint 1 (solid): w + b = 4Constraint 2 (dashed): 2w + b = 6⊕Unique solution (circled plus)

Current constraints and a semantic alternative to the graph

Constraint	Equation	What it contributes
Constraint 1 (solid)	w + b = 4	Defines the first set of parameter pairs that fit.
Constraint 2 (dashed)	2w + b = 6	Crosses the first line once and fixes both parameters.
Exact classification	Unique and consistent
Solution set	{(2, 2)}
Meaning for the model parameters	The observations identify weight w = 2 and bias b = 2.

The first server-rendered state is the unique system $w+b=4$ and $2w+b=6$, including its status, graph, and semantic table. Selecting the noisy case adds the three constraints created by observations $(0,2)$, $(1,4)$, and $(2,6.1)$ for the model $\widehat y=wx+b$. The first two select $(w,b)=(2,2)$, but the third requires $2(2)+2=6.1$, which is false.

A coefficient test that does not divide by zero

For two nondegenerate equations

$$\begin{aligned} a_1u+b_1v&=c_1,\\ a_2u+b_2v&=c_2, \end{aligned}$$

compare the coefficient pairs $(a_1,b_1)$ and $(a_2,b_2)$. They are proportional when one is a real multiple of the other: there is a number $k$ such that

$$(a_2,b_2)=k(a_1,b_1).$$

This formulation is safer than separately comparing $a_2/a_1$ and $b_2/b_1$, because either denominator might be zero.

Coefficient relationship	Right-side relationship	Geometry	Solutions
No multiplier $k$ matches both coefficients	Not needed	Nonparallel lines	Exactly one
$(a_2,b_2)=k(a_1,b_1)$	$c_2\ne kc_1$	Distinct parallel lines	None
$(a_2,b_2)=k(a_1,b_1)$	$c_2=kc_1$	The same line twice	Infinitely many

The unique case means the two equations constrain different directions. The parallel cases have the same left-side pattern; scaling the first left side either reproduces the second right side or exposes a contradiction.

Zero coefficients remain ordinary data. The system

$$\begin{aligned} 0u+v&=-1,\\ u+0v&=3 \end{aligned}$$

is simply $v=-1$ and $u=3$. A horizontal line and a vertical line meet at the unique point $(3,-1)$. Ratio tests that try to divide by $a_1=0$ are the problem; the zero coefficient is not.

If an entire left side is zero, the equation is degenerate and must be read directly:

Degenerate equation	Meaning	Effect on a larger system
$0u+0v=0$	An identity true for every real pair	Adds no restriction
$0u+0v=5$	A contradiction true for no real pair	Makes the complete system inconsistent

For example, $u+v=4$ together with $0=0$ still leaves the infinite family $(t,4-t)$. The identity does not upgrade one line into a unique point.

Identifiability asks what the observations add

Return to $\widehat y=wx+b$. Each observation $(x_i,y_i)$ becomes one parameter constraint

$$x_iw+b=y_i.$$

For observations $(1,4)$ and $(3,8)$, the constraints are

$$\begin{aligned} w+b&=4,\\ 3w+b&=8. \end{aligned}$$

Subtracting yields $2w=4$, so $w=2$ and $b=2$. The different input values made the observations constrain the parameters differently.

Repeating $(1,4)$ one hundred times still writes $w+b=4$ one hundred times. The dataset has more rows, but the allowed parameter family remains $(w,b)=(t,4-t)$. Those observations do not identify $w$ and $b$ separately.

If the repeated input carries two targets, such as $(1,4)$ and $(1,6)$, the constraints become $w+b=4$ and $w+b=6$. The exact affine model cannot satisfy both. This is an exact-fit diagnosis, not proof that the data are worthless. Measurement noise, rounded labels, or a model that is too simple can all make real observations inconsistent.

Later lessons will introduce rank as vocabulary for counting independent linear information. The present lesson needs only the visible test: do the constraints cross, conflict, or repeat?

NumPy separates exact solving from approximate fitting

The unique worked system can be written as an array equation. Each row of A contains the coefficients multiplying $(u,v)$, and each entry of c contains the required right side:

import numpy as np

A = np.array([
    [2.0,  1.0],
    [1.0, -1.0],
])
c = np.array([7.0, 2.0])

solution = np.linalg.solve(A, c)
verified = np.allclose(A @ solution, c, rtol=0.0, atol=1e-12)

print("solution:", solution)
print("A @ solution:", A @ solution)
print("passes both equations:", verified)

# Three slightly inconsistent observations of y_hat = w*x + b.
x_observed = np.array([0.0, 1.0, 2.0])
y_observed = np.array([2.0, 4.0, 6.1])
design = np.column_stack((x_observed, np.ones_like(x_observed)))

approximate, _, _, _ = np.linalg.lstsq(design, y_observed, rcond=None)
predictions = design @ approximate
residuals = predictions - y_observed

print("approximate [w, b]:", np.round(approximate, 6))
print("predictions:", np.round(predictions, 6))
print("residuals:", np.round(residuals, 6))
print(
    "all three exact:",
    np.allclose(residuals, 0.0, rtol=0.0, atol=1e-12),
)

The verified output is:

solution: [3. 1.]
A @ solution: [7. 2.]
passes both equations: True
approximate [w, b]: [2.05     1.983333]
predictions: [1.983333 4.033333 6.083333]
residuals: [-0.016667  0.033333 -0.016667]
all three exact: False

The official numpy.linalg.solve documentation specifies an exact numerical solve for a square system with a unique solution; it raises LinAlgError for a singular or nonsquare coefficient array. That error alone cannot distinguish “no solution” from “infinitely many,” because both parallel configurations fail the unique-solution requirement. Algebraic classification must supply the meaning.

numpy.allclose checks arrays within chosen tolerances. The code sets rtol=0.0 and an explicit absolute tolerance because the reference residual is zero; NumPy warns that its default absolute tolerance is not suitable for every small-number scale. This numerical check supports the exact derivation but does not redefine mathematical equality.

The noisy rows have no shared exact parameter pair, so the second calculation asks a different question. numpy.linalg.lstsq documentation describes the returned vector as minimizing the Euclidean size of the residual when an exact solution is unavailable. Its $(w,b)\approx(2.05,1.9833)$ produces small residuals, not zero residuals. Least squares will receive its own later lesson; here it marks the boundary between solving every constraint and approximating incompatible observations.

The next row should change the intersection

Classifying a system is ultimately a question about what each constraint adds. A crossing fixes a single point. A contradiction empties the solution set. A scaled repeat leaves a free choice. For an AI model, those outcomes translate to identifiable parameters, an impossible exact fit, or parameters the data cannot distinguish.

The next lesson will give vectors a formal role as representations of AI data. Keep the diagnostic from this lesson close: before trusting a recovered parameter, check whether the observations actually selected it uniquely.

Check your understanding

Question 1

Solve 2x + y = 8 and x − y = 1. Show why the system has exactly one solution and verify the point in both equations.

Show the step-by-step solution

Add the equations so that $y$ cancels:

$$(2x+y)+(x-y)=8+1 \iff 3x=9 \iff x=3.$$

Substitute $x=3$ into $x-y=1$:

$$3-y=1 \iff y=2.$$

Now test $(3,2)$ in both originals:

$$2(3)+2=8, \qquad 3-2=1.$$

The elimination fixed $x$, substitution fixed $y$, and the point passes both truth tests. The solution set is therefore exactly $\{(3,2)\}$.

Question 2

Classify x + 3y = 5 and 2x + 6y = 13. Derive the decisive statement and interpret its geometry.

Show the step-by-step solution

Twice the first left side is $2x+6y$. Subtract twice the first equation from the second:

$$(2x+6y)-2(x+3y)=13-2(5) \iff 0=3.$$

The contradiction cannot hold for any real $x$ or $y$. The coefficient pair $(2,6)$ is twice $(1,3)$, but the right side $13$ is not twice $5$. The lines are parallel and distinct, so the system is inconsistent and its solution set is $\varnothing$.

Question 3

Find every solution of 4p − 2q = 6 and −2p + q = −3. Explain why 0 = 0 is not the final answer.

Show the step-by-step solution

Multiplying the second equation by $-2$ gives

$$-2(-2p+q)=-2(-3) \iff 4p-2q=6,$$

which is the first equation. Eliminating one with the other therefore yields $0=0$, showing that the equations repeat the same restriction.

One restriction remains. Divide $4p-2q=6$ by $2$:

$$2p-q=3 \iff q=2p-3.$$

Choose $p=t$ for any $t\in\mathbb R$. Then every solution is

$$(p,q)=(t,2t-3).$$

The identity $0=0$ diagnosed redundancy; the parameterized line, not the identity alone, is the complete infinite solution set.

Question 4

Without dividing by a zero coefficient, classify 0x + y = 4 and 3x + 0y = 9.

Show the step-by-step solution

Read the zero coefficients directly. The first equation is $y=4$, a horizontal line. The second is $3x=9$, so division by the nonzero number $3$ gives $x=3$, a vertical line.

Those lines meet once at $(x,y)=(3,4)$. Verification gives

$$0(3)+4=4, \qquad 3(3)+0(4)=9.$$

The system is consistent and unique. No ratio such as $3/0$ is needed.

Question 5

What happens to the system x + y = 2 if a second equation is (a) 0x + 0y = 0 or (b) 0x + 0y = 7? Give each solution set.

Show the step-by-step solution

For part (a), the second equation is $0=0$, true for every real pair. It adds no restriction. The first line remains, so choose $x=t$ and obtain $y=2-t$:

$$S_a=\{(t,2-t)\mid t\in\mathbb R\}.$$

The system is consistent and underdetermined, with infinitely many solutions.

For part (b), the second equation is $0=7$, false for every real pair. A solution would have to satisfy that contradiction as well as the first line, which is impossible. Therefore

$$S_b=\varnothing,$$

and the system is inconsistent.

Question 6

The model ŷ = wx + b receives the observation (2, 7) fifty times. Are w and b identifiable? Parameterize every exact fit and explain what the repeated rows add.

Show the step-by-step solution

Every repeated observation produces the same constraint:

$$2w+b=7.$$

Choose the weight freely as $w=t$. Then the bias must be

$$b=7-2t.$$

Thus all exact fits are

$$(w,b)=(t,7-2t), \qquad t\in\mathbb R.$$

Fifty copies still draw one parameter line, so infinitely many pairs fit. The rows confirm the same observation but add no distinct restriction. Neither parameter is identifiable from this dataset alone.

Question 7

For observations (0, 2), (1, 4), and (2, 6.1) under ŷ = wx + b, prove there is no exact fit. Then explain what the NumPy least-squares output [2.05, 1.983333] does and does not claim.

Show the step-by-step solution

The first observation forces $b=2$. The second then gives

$$w+b=4 \iff w+2=4 \iff w=2.$$

The only pair satisfying the first two rows is $(w,b)=(2,2)$. Test it on the third:

$$2w+b=2(2)+2=6\ne6.1.$$

Therefore no parameter pair satisfies all three constraints exactly.

The least-squares output uses $w\approx2.05$ and $b\approx1.983333$. Its predictions are approximately $(1.983333,4.033333,6.083333)$, leaving residuals approximately $(-0.016667,0.033333,-0.016667)$. Those values are small but nonzero. The output minimizes a specified residual measure; it does not turn the inconsistent exact system into a consistent one or claim that every observation is matched exactly.

Sources

1. Mathematics for Machine Learning companion website
2. Mathematics for Machine Learning book PDF
3. NumPy documentation: numpy.linalg.solve
4. NumPy documentation: numpy.linalg.lstsq
5. NumPy documentation: numpy.allclose