Words Become Geometry

The astonishing leap behind Word2Vec is not merely that a machine can attach numbers to words. It is that meaning begins to acquire shape.

Suppose every word is represented by a vector $v_w \in \mathbb{R}^d$. Words appearing in similar linguistic neighbourhoods are trained toward nearby locations.

$$\operatorname{similarity}(u,v)=\frac{u\cdot v}{\lVert u\rVert\lVert v\rVert}=\cos\theta.$$

This gives us the famous, approximate relational pattern:

$$v_{\text{king}}-v_{\text{man}}+v_{\text{woman}}\approx v_{\text{queen}}.$$

The deeper surprise is that language has not merely been catalogued. Through patterns of use, it has been persuaded to reveal a geometry.

A tiny experiment

similarity = cosine(embedding["music"], embedding["melody"])
print(similarity)

The number matters less than the picture it suggests: concepts arranged as a landscape, with analogy behaving like translation.