Distance and Cosine Similarity | COSMO - Community of Science and Math Outreach

Two vectors can be close in one sense and far apart in another. Choosing a similarity rule is therefore part of the model, not a detail added after the representation is built.

For vectors $\mathbf{x},\mathbf{y}\in\mathbb{R}^d$ , Euclidean distance is

\lVert\mathbf{x}-\mathbf{y}\rVert_2= \sqrt{(x_1-y_1)^2+\cdots+(x_d-y_d)^2}.

It measures the straight-line distance between two points. Cosine similarity is

\frac{\mathbf{x}\cdot\mathbf{y}} {\lVert\mathbf{x}\rVert_2\lVert\mathbf{y}\rVert_2},

which is the cosine of the angle between them. It equals $1$ for vectors pointing in the same direction, $0$ for perpendicular vectors, and $-1$ for opposite directions.

A ranking disagreement

Let $q=(1,1)$ , $a=(2,2)$ , and $b=(1,0)$ . Cosine similarity ranks $a$ above $b$ because $a$ points in exactly the same direction as $q$ :

\cos(q,a)=1,\qquad \cos(q,b)=\frac{1}{\sqrt 2}.

Euclidean distance gives

d(q,a)=\sqrt 2,\qquad d(q,b)=1,

so it ranks $b$ as closer. Neither answer is automatically wrong. If length records confidence or amount, Euclidean distance may preserve useful information. If length mainly reflects document size, cosine similarity may be more appropriate.

Why normalization connects the two

Normalize each nonzero vector to unit length:

\hat{\mathbf{x}}=\frac{\mathbf{x}}{\lVert\mathbf{x}\rVert_2}.

Then

\lVert\hat{\mathbf{x}}-\hat{\mathbf{y}}\rVert_2^2 =2-2\cos(\theta).

The derivation follows by expanding the squared norm:

(\hat{\mathbf{x}}-\hat{\mathbf{y}})\cdot(\hat{\mathbf{x}}-\hat{\mathbf{y}}) =1+1-2\hat{\mathbf{x}}\cdot\hat{\mathbf{y}}.

Therefore, on unit-normalized vectors, ranking by smallest Euclidean distance is equivalent to ranking by largest cosine similarity. This fact is useful when a search system supports one metric more efficiently than another.

Metric questions

Euclidean distance is a metric: it is nonnegative, symmetric, zero only for identical points, and obeys the triangle inequality. Cosine similarity is a similarity score rather than a distance metric. The quantity $1-\cos(\theta)$ is often called cosine distance, but it does not satisfy every metric property in every setting.

That distinction matters when an algorithm relies on the triangle inequality to avoid unnecessary comparisons. It matters less when the system simply scores every candidate.

A useful classroom test

Plot a query point and several candidates in two dimensions. Ask students to predict the Euclidean and cosine rankings before calculating them. Then multiply one candidate by $10$ . Its direction stays fixed, so cosine similarity does not change, while Euclidean distance usually changes greatly.

Follow with three questions:

Should magnitude matter for this data?
Were the coordinates measured on comparable scales?
Does the chosen score produce useful neighbors on actual examples?

The final question is decisive. Geometry supplies several reasonable comparisons; evaluation tells us which geometry fits the task.