Dot product, norms, and similarity

The idea

The dot product is the one operation underneath almost every notion of similarity in data science. Computationally it is trivial:

$a \cdot b = \sum_{i=1}^{d} a_i b_i$

Geometrically it is a signed alignment score:

$a \cdot b = \|a\| \, \|b\| \cos\theta$

where $\theta$ is the angle between the vectors. Same direction: large and positive. Orthogonal: zero. Opposite: negative. Two things scale it — how long the vectors are, and how aligned they are. Every similarity measure you use is a choice about which of those two factors you want to keep.

From the dot product you get norms (size):

$\|a\|_2 = \sqrt{a \cdot a} = \sqrt{\textstyle\sum_i a_i^2}, \qquad \|a\|_1 = \textstyle\sum_i |a_i|$

and Euclidean distance (separation), tied back to the dot product by an identity worth memorizing:

$\|a - b\|^2 = \|a\|^2 + \|b\|^2 - 2 \, a \cdot b$

Cosine similarity strips magnitude out and keeps only alignment:

$\text{cos-sim}(a, b) = \frac{a \cdot b}{\|a\| \, \|b\|} = \cos\theta \in [-1, 1]$

The decision rule between them: is magnitude signal or nuisance? Document length, purchase counts, and embedding norms are usually nuisance — use cosine. Sensor readings, standardized features, and coordinates are usually signal — use Euclidean. And the two choices are secretly one after normalization: for unit vectors, $\|a - b\|^2 = 2 - 2\cos\theta$ , so Euclidean ranking on L2-normalized vectors is cosine ranking. That is why the standard production pattern is "normalize once at index time, then use whichever the index computes fastest."

One more reusable tool: the projection of $a$ onto $b$ ,

$\text{proj}_b(a) = \frac{a \cdot b}{\|b\|^2} \, b$

is "the part of $a$ that lies along $b$ " — the geometric primitive behind least squares, PCA reconstruction, and orthogonalization.

Drag the two vectors below and watch dot product and cosine similarity respond differently to length versus direction:

aₓ = 3.0aᵧ = 1.0bₓ = 1.0bᵧ = 2.4

Dot product

5.40

Cosine similarity

0.66

Angle

49°

Drag either vector: stretching one along its own direction moves the dot product but leaves cosine similarity unchanged — magnitude is nuisance, alignment is signal.

Worked example

Word-count vectors over the vocabulary (data, model, pizza):

Doc	Vector	Description
$A$	$(2, 1, 0)$	short ML article
$B$	$(4, 2, 0)$	the same article pasted twice
$C$	$(0, 1, 2)$	restaurant review

Compute similarities of $B$ and $C$ against $A$ :

For $B$ against $A$ :

\begin{aligned} A \cdot B &= 2\cdot4 + 1\cdot2 = 10, \\ \lVert A\rVert &= \sqrt{5} \approx 2.24, & \lVert B\rVert &= \sqrt{20} \approx 4.47, \\ \cos(A,B) &= \frac{10}{\sqrt{5}\sqrt{20}} = 1.0, \\ \lVert A-B\rVert &= \sqrt{(-2)^2+(-1)^2} = \sqrt{5} \approx 2.24. \end{aligned}

For $C$ against $A$ :

\begin{aligned} A \cdot C &= 1, & \lVert C\rVert &= \sqrt{5}, \\ \cos(A,C) &= \frac{1}{\sqrt{5}\sqrt{5}} = 0.2, \\ \lVert A-C\rVert &= \sqrt{4+0+4} = \sqrt{8} \approx 2.83. \end{aligned}

Read the punchline off the numbers: Euclidean distance claims the pasted duplicate $B$ (distance $2.24$ ) is nearly as far from $A$ as the pizza review $C$ (distance $2.83$ ). Cosine correctly says $B$ is the same document ( $1.0$ ) and $C$ is off-topic ( $0.2$ ). Magnitude — here, document length — was pure nuisance, and Euclidean distance charged for it.

Now normalize everything to unit length and recompute Euclidean distance: $\hat{A} = \hat{B} \approx (0.894, 0.447, 0)$ , so $\|\hat{A} - \hat{B}\| = 0$ , and $\|\hat{A} - \hat{C}\|^2 = 2 - 2(0.2) = 1.6$ . The normalized Euclidean ranking now agrees with cosine exactly, as the identity promises.

Cheat sheet

Core formulas

Quantity	Formula	Notes
Dot product	$a \cdot b = \sum_i a_i b_i = \lVert a \rVert \lVert b \rVert \cos\theta$	signed alignment
L2 norm	$\lVert a \rVert_2 = \sqrt{a \cdot a}$	rotation-invariant
L1 norm	$\lVert a \rVert_1 = \sum_i \lvert a_i \rvert$	robust to one big coordinate
Euclidean distance	$\lVert a - b \rVert_2$	squared: $\lVert a - b \rVert_2^2 = \lVert a \rVert^2 + \lVert b \rVert^2 - 2 a \cdot b$
Cosine similarity	$\frac{a \cdot b}{\lVert a \rVert \lVert b \rVert} \in [-1, 1]$	magnitude-blind
Projection of $a$ on $b$	$\frac{a \cdot b}{\lVert b \rVert^2} b$	basis of least squares and PCA

Decision rules

Magnitude is signal (measurements, scaled features) → Euclidean.
Magnitude is nuisance (counts, text, embeddings) → cosine.
Can normalize at index time → do it; then the two rankings coincide via $\|a-b\|^2 = 2 - 2\cos\theta$ on the unit sphere.
Mixed-scale raw features + any distance method (k-NN, k-means) → standardize first or one feature silently dominates.

Gotchas

Cosine similarity is undefined for the zero vector — guard empty documents and all-zero embeddings.
Cosine distance ( $1 - \cos\theta$ ) violates the triangle inequality — the angle $\theta$ itself is the metric, if you need one.
$a \cdot b = 0$ means orthogonal, not "opposite" — opposite is $\cos\theta = -1$ .
Raw dot-product similarity (no normalization) favors long vectors — in recommenders this systematically over-ranks popular items.
In high dimensions, random vectors are nearly orthogonal and distances concentrate — neighbor contrast comes from real structure, not from the geometry being kind.