Dot Products Turn Features Into Scores

An AI model can print a score such as $1.5$ without showing which input features made the number rise or fall. A dot product opens that calculation one small piece at a time: multiply each feature by the weight assigned to the same coordinate, then add the products.

By the end of this lesson, you will be able to expand a dot product into signed feature contributions, implement it with NumPy, and read the raw score inside a binary linear classifier without mistaking it for a probability.

Learning outcome: Interpret a dot product as a weighted feature score.

This lesson builds on Lesson 041: Inner Products Measure Alignment, where the same operation described how two vectors point relative to one another. Section 3.2 of Mathematics for Machine Learning provides the mathematical reference. The feature names, numbers, code, visual, classifier walkthrough, and assessments here are original.

A dot product pairs features with weights

Let $x=(x_1,\ldots,x_n)$ be a feature vector: one fixed-order list of numbers that represents an input. Let $w=(w_1,\ldots,w_n)$ be a weight vector: one fixed-order list of parameters that says how the model combines those features. Both vectors have $n$ coordinates, and coordinate $i$ must mean the same thing in both lists.

The transpose notation makes the shape check visible. If $w$ is treated as an $n\times1$ column, then $w^{\mathsf T}$ is $1\times n$ and $w^{\mathsf T}x$ is a $1\times1$ result—a scalar score. The summation says the same thing in a form that is easier to inspect: there is one multiplication for each coordinate, followed by one addition chain.

The order is part of the data contract. If $x_2$ is the error rate, then $w_2$ must be the error-rate weight. Two arrays can have the same length and still describe the wrong model if their coordinates have been permuted.

The dot product is also the standard real inner product from the previous lesson, but the roles are different here. One vector is an input representation; the other is a learned or chosen set of scoring parameters. That role assignment makes each product $w_i x_i$ useful to inspect.

A worked score makes each contribution visible

Use a small feature vector and weight vector:

$$x=(2.0,\ 0.5,\ 4.0,\ 3.0),\qquad w=(1.2,\ -0.8,\ 0.0,\ 0.4).$$

The four coordinates are already numeric inputs to the model. Their names are only labels for this example; in a real pipeline, preprocessing determines what each number means and what scale it uses.

The contribution of coordinate $i$ is $w_i x_i$. It is positive when the feature and weight have the same sign, negative when their signs differ, and zero when either one is zero. A positive contribution pushes the score upward; a negative contribution pulls it downward. The magnitude $\lvert w_i x_i\rvert$ tells how large that coordinate’s direct effect is in this particular score.

Trace each feature product to the dot-product total

The table is the complete numerical fallback for the contribution diagram.

Feature	Value xᵢ	Weight wᵢ	Contribution wᵢxᵢ
Coverage	2.00	1.20	positive +2.40
Error indicator	0.50	−0.80	negative −0.40
Reference deviation	4.00	0.00	zero 0.00
Verified signal	3.00	0.40	positive +1.20
Dot product total			3.20

Read the table from top to bottom to follow every value, weight, sign, and contribution without relying on the diagram’s colors or geometry.

The table is the authoritative fallback: the contribution diagram adds a quick visual trace of the running total but does not hide any number in color, hover text, or client-side state. The zero-weight feature contributes nothing to this model score even though its feature value is $4.0$.

If the model also has a bias $b$, keep that offset separate from the feature products. With $b=-1.7$, the affine score becomes

$$s=w^{\mathsf T}x+b=3.2-1.7=1.5.$$

The bias shifts every input’s score by the same amount. It is not evidence that one of the four features contributed $-1.7$.

Why the coordinate products are useful—but limited

The sum is exactly additive. If only feature $i$ changes by $\Delta$, then

$$w^{\mathsf T}(x+\Delta e_i)=w^{\mathsf T}x+w_i\Delta,$$

where $e_i$ is the vector that is $1$ in position $i$ and $0$ elsewhere. So a fixed linear score changes by $w_i\Delta$—there is no hidden interaction term between that change and the other coordinates.

That makes $w_i x_i$ a useful explanation of this one linear calculation. It does not make the product a complete causal attribution. A trained system may have correlated features, preprocessing steps, or later nonlinear layers. Also, the size of a weight is not comparable across features with different units or scales. Standardizing features can change the numerical weights while leaving the model’s predictions equivalent after the corresponding parameter update.

A linear classifier thresholds the score

A binary linear classifier uses the affine score

$$g(x)=w^{\mathsf T}x+b$$

and then applies a decision rule such as

$$\hat y= \begin{cases} 1 & \text{if }g(x)\ge 0,\\ 0 & \text{if }g(x)<0. \end{cases}$$

For the worked input, $g(x)=1.5$, so a threshold at zero assigns class $1$. The threshold is a second step; the dot product itself only supplies the weighted score. Changing the threshold changes the decision without changing the feature products.

The set of points with $g(x)=0$ is a line in two dimensions and a hyperplane in higher dimensions. The weight vector $w$ is perpendicular to that boundary under the standard Euclidean geometry. Moving an input in a direction that aligns with $w$ raises the score; moving against it lowers the score. This is why a dot product turns a list of features into a position relative to a classifier boundary.

The distinction also appears in scikit-learn. Its linear-model guide writes a prediction as an intercept plus a weighted combination of features (linear models), while LogisticRegression.decision_function returns confidence scores related to signed distance from the hyperplane (LogisticRegression documentation). Those scores are not probabilities. A logistic model can apply a sigmoid,

$$\sigma(g)=\frac{1}{1+e^{-g}},$$

which maps $g=1.5$ to approximately $0.818$. That value is a probability estimate produced by the model’s link function; it is not what the dot product means by itself, and whether an estimate is well calibrated is a separate question. In the same documentation, predict_proba is the API that returns probability estimates.

The same operation can mean compatibility in attention

In an attention calculation, a query $q$ and a key $k$ are both representation vectors. Their dot product $q^{\mathsf T}k$ is a compatibility score used to compare that key with other keys for the query. The arithmetic is unchanged: pair matching coordinates, multiply, and sum.

The interpretation is different from the feature example. In $w^{\mathsf T}x$, $w$ plays the parameter role and $x$ plays the input role, so $w_i x_i$ is a coordinate contribution to one model score. In $q^{\mathsf T}k$, both vectors come from representations, so a product is one part of a compatibility calculation—not automatically a causal explanation of why a token matters. Lesson 041 follows this score through scaling and softmax; the raw dot product still comes first.

NumPy keeps the implementation close to the algebra

For one-dimensional arrays, numpy.dot computes the inner product of the two vectors and returns a scalar. The elementwise multiplication is useful when the goal is to inspect contributions before summing them:

import numpy as np

x = np.array([2.0, 0.5, 4.0, 3.0])
w = np.array([1.2, -0.8, 0.0, 0.4])
b = -1.7

contributions = w * x
raw_score = np.dot(w, x)
classifier_score = raw_score + b

assert np.allclose(contributions.sum(), raw_score)
print(contributions)
print(raw_score)
print(classifier_score)

The output is:

[ 2.4 -0.4  0.   1.2]
3.2
1.5

The assert checks the central identity: summing the visible coordinate products agrees with the compact dot-product call. Keeping both forms in a debugging notebook makes a score easier to audit than treating dot as a black box.

Boundary cases catch plausible-looking mistakes

Four checks prevent most beginner errors:

• Same shape is not the same meaning. If the feature order is changed but the weight order is not, NumPy can still multiply arrays of shape $(4,)$. For example, pairing the weights above with $x'=(2.0,4.0,0.5,3.0)$ gives $2.4-3.2+0.0+1.2=0.4$, not $3.2$. The code runs; the model input is wrong.
• A zero vector gives a zero dot product. If every feature is zero, then $w^{\mathsf T}x=0$ for every $w$. With a bias, the classifier score is still $b$. A zero score therefore does not prove that all features canceled each other, and a zero vector has no direction for an angle-based interpretation.
• Negative is not automatically bad. A negative feature and a negative weight multiply to a positive contribution. For example, $(-2)(-0.7)=+1.4$. The sign describes the product in this model, not a moral label for the feature.
• A score is not a probability. A raw value can be $-4$, $0$, or $12$. Only an explicitly chosen output rule—such as a threshold, sigmoid, or softmax—changes that score into a decision or a normalized estimate.

The next lesson moves from one weighted score to a different geometric question: how far apart are two points when the chosen notion of size changes? The Math for AI course page lists Lesson 043: Distance depends on the chosen geometry as the next step.

Check your understanding

Question 1

Show the step-by-step solution

Let $x=(2,-1,3)$ and $w=(0.5,2,-1)$. Compute the three coordinate contributions and the dot product. Which term lowers the score?

The matching products are

$$0.5(2)=1,\qquad 2(-1)=-2,\qquad (-1)(3)=-3.$$

Add them in coordinate order:

$$w^{\mathsf T}x=1+(-2)+(-3)=-4.$$

The second and third terms lower the total because they are negative. The third term has the larger downward magnitude, $3$ rather than $2$.

Question 2

Show the step-by-step solution

For $x=(3,2,-4)$ and $w=(-2,0.5,0)$, calculate the score and explain why the third feature does not change it.

Multiply coordinate by coordinate:

$$(-2)(3)=-6,\qquad 0.5(2)=1,\qquad 0(-4)=0.$$

The score is therefore

$$w^{\mathsf T}x=-6+1+0=-5.$$

The third feature has value $-4$, but its weight is zero, so its product is zero. For this fitted score, that coordinate is ignored; this does not mean the underlying signal could never matter in another model or after retraining.

Question 3

Show the step-by-step solution

A classifier has $x=(2,-1)$, $w=(1,-2)$, and $b=-4$. Compute its affine score and apply a threshold at zero.

First compute the dot product:

$$w^{\mathsf T}x=1(2)+(-2)(-1)=2+2=4.$$

Then add the bias:

$$g(x)=4+(-4)=0.$$

The score is exactly on the threshold. With the rule $g(x)\ge 0\Rightarrow \hat y=1$, this input is assigned class $1$. A rule using a strict $>$ instead would make the boundary convention explicit and could assign a different class.

Question 4

Show the step-by-step solution

The correct feature order is $x=(2,1,4)$ with weights $w=(1,-3,2)$. A bug passes $x'=(2,4,1)$ instead. Compute both scores and identify the failure.

With the intended order,

$$w^{\mathsf T}x=1(2)+(-3)(1)+2(4)=2-3+8=7.$$

With the permuted feature vector,

$$w^{\mathsf T}x'=1(2)+(-3)(4)+2(1)=2-12+2=-8.$$

Both arrays have shape $(3,)$, so a shape check alone would not catch the mistake. The products pair weights with the wrong feature meanings; the data schema must preserve coordinate order as well as array shape.

Question 5

Show the step-by-step solution

A centered feature has value $x_i=-2$ and its learned weight is $w_i=-0.7$. Does this coordinate lower the score?

Compute the product rather than judging from the feature sign alone:

$$w_i x_i=(-0.7)(-2)=+1.4.$$

It raises the score by $1.4$. The two negative signs cancel. In a centered or otherwise transformed feature space, negative feature values are ordinary numbers, not errors.

Question 6

Show the step-by-step solution

A raw classifier score is $g(x)=1.5$. Is $1.5$ a probability? If the model explicitly uses the sigmoid link, what number does it produce before any calibration discussion?

The raw score is not a probability: it can be outside the interval $[0,1]$. The sigmoid gives

$$\sigma(1.5)=\frac{1}{1+e^{-1.5}}\approx0.8176.$$

So the link function produces approximately $0.818$. Calling that a useful probability estimate requires the model and its calibration assumptions; the dot product alone only produced the score $1.5$.

Question 7

Show the step-by-step solution

In NumPy, x.shape == (4,) and w.shape == (4,), but a score suddenly changes after a data-pipeline refactor. Give one numerical check that can reveal a coordinate-order bug using the worked example.

Recompute the elementwise products and compare their sum with the expected ordered score:

$$w*x=(2.4,-0.4,0.0,1.2),\qquad \sum_i w_i x_i=3.2.$$

Then print the feature names beside the values before the dot product. If the pipeline produced $(2.0,4.0,0.5,3.0)$ while the weights stayed in their original order, the visible products become $(2.4,-3.2,0.0,1.2)$ and sum to $0.4$. The arrays still have compatible shapes, so a contribution table or named feature-order assertion is needed in addition to the shape check.

Question 8

Show the step-by-step solution

An attention-style comparison uses $q=(1,2,-1)$ and $k=(2,0,1)$. Compute $q^{\mathsf T}k$ and explain one interpretation difference from $w^{\mathsf T}x$ in a classifier.

Pair and add:

$$q^{\mathsf T}k=1(2)+2(0)+(-1)(1)=2+0-1=1.$$

The result is a compatibility score between a query and a key. In a classifier, $w$ is the parameter vector and $x$ is the input, so each $w_i x_i$ is a coordinate contribution to a feature score. In attention, both $q$ and $k$ are representations; a coordinate product contributes to their comparison but is not automatically a causal feature attribution.

Sources

1. Mathematics for Machine Learning book PDF, Section 3.2
2. NumPy documentation: numpy.dot
3. scikit-learn documentation: Linear Models
4. scikit-learn documentation: LogisticRegression