Word Embeddings and Word2Vec — ELI5

Computers do not understand words. They understand numbers. So before a computer can do anything useful with text, it needs a way to turn words into numbers.

The simplest way is to number the words: “cat” is 1, “dog” is 2, “fish” is 3. But this is useless because the numbers say nothing about meaning. The computer sees “cat” (1) as closer to “dog” (2) than to “fish” (3), but only because of random numbering — not because cats and dogs are both pets.

Word2Vec solves this by turning each word into a list of numbers — typically 100 to 300 numbers. Think of it as giving each word a home address in a neighborhood. Words that mean similar things live on the same block. “Cat” and “kitten” are neighbors. “Paris” and “London” are on the same street. “Happy” and “joyful” share a porch.

How does Word2Vec figure out who goes where? It reads millions of sentences and notices which words appear in similar surroundings. “Cat” and “dog” both show up near “pet,” “feed,” “vet,” and “cute.” Since they keep the same company, Word2Vec places them close together.

The most famous trick is word math. Take the numbers for “king,” subtract “man,” add “woman,” and you get something very close to “queen.” The computer stumbled onto the idea that royalty and gender are separate concepts — just from reading text.

A common mix-up is thinking Word2Vec understands meaning. It does not. It learned statistical patterns about which words hang out together. It is a map of word neighborhoods, not a brain.

The one thing to remember: Word2Vec turns words into number lists where similar words get similar numbers, letting computers work with meaning instead of just spelling.