What a 2D Plot of Your Embeddings Can (and Can't) Tell You

Hundreds of dimensions, one flat picture

Text embedding models typically output vectors with hundreds or thousands of numbers per piece of text — far beyond anything a human can look at directly and reason about. Projecting those vectors down to two dimensions for a scatter plot is a natural instinct: if similar concepts are supposed to sit near each other in embedding space, a 2D plot should show that visually.

How the projection actually works

PCA (principal component analysis) finds the two directions in your specific dataset that capture the most variance, and plots each point's position along just those two directions. It's a linear method: fast, deterministic, and genuinely good at surfacing dominant, large-scale structure like well-separated clusters or a strong single axis of variation in your data.

Where it can mislead you

Because PCA only keeps two directions out of potentially thousands, it necessarily throws away information. Two points that land close together in the 2D plot are not guaranteed to be close in the true high-dimensional space — their similarity along the two kept directions could be coincidental while they differ substantially along directions the plot discarded. The reverse can happen too: genuinely similar vectors can appear more separated in a 2D projection than they really are, if their similarity lives mostly outside the top two principal directions.

What it's good for (and what it isn't)

Treat a PCA plot as a fast, zero-setup sanity check — "do my categories separate into visually distinct groups at all?" — not as a precise map of true similarity. For deeper, more faithful analysis of complex embedding structure, non-linear techniques like t-SNE or UMAP are worth reaching for, at the cost of more setup and computation.

Try it on your own vectors

Our Embedding Vector Visualizer runs this exact PCA projection entirely in your browser — paste in vectors of any dimension and get an instant 2D plot, verified against synthetic clustered data to confirm well-separated groups stay visually separated after projection.