Cosine similarity

Comparing two embeddings by the angle between them, and why the number it gives back is not a percentage.

Part of the Memory and truth track on lAItest.

A long report and a one-line summary of it sit far apart in the space. They should still count as a match.

Which is why retrieval measures the angle and throws away the length.

Cosine similarity measures direction, not distance.

Draw an arrow from the origin out to each point. Ignore how long the arrows are. Ask only how much they point the same way. Pointing identically scores 1, at right angles 0, opposite −1. Length usually tracks how much text there is, and how much text there is should not decide whether two things mean the same thing.

Try it

Drag a word straight out from the centre: its neighbours do not change. Drag it sideways and they do. This step is an interactive widget; open the lesson to use it.

A common misconception

Commonly believed: 0.82 means an 82% match, so anything under 0.8 can be thrown away.

Actually: The scale is uncalibrated. Every embedding model spreads its scores differently, so 0.82 from one model and 0.82 from another are not the same claim, and neither is a probability. A cut-off tuned against one model can quietly break the day you change models. Ranking the results and taking the top few is far more portable than trusting an absolute line.

You switch to a newer embedding model and keep your existing 0.75 similarity cut-off. What is the risk?

Answer: The new model spreads its scores differently, so the cut-off now keeps too much or too little. Cosine scores are not calibrated across models, so a threshold is a property of the model you tuned it on. Option four hides the other half of the trap: vectors from two different models generally cannot be compared at all, so a model change usually means re-embedding the whole corpus. A few families are deliberately built to share one space across their own sizes; across vendors, assume not.

In one sentence

Cosine similarity tells you which result is closer. It does not tell you whether any of them is good.