Open any music app and hit "radio" on a song you like. It plays songs you've never heard — and they fit. Nobody tagged them "similar to this one". Somewhere, every song became a list of numbers, and the app simply found nearby numbers. That trick — turning meaning into coordinates — is the single idea that makes RAG possible, and it's called an embedding. Get this lesson properly and the next four become obvious.
The problem with matching words
A user asks: "how many CLs do I get?" Your handbook says: "Employees are entitled to 12 casual leaves per calendar year." Count the shared words. "I". "get". That's it — and neither helps.
Traditional search would need a human to maintain a synonym list: CL = casual leave, PTO = paid time off, "won't start" = "fails to boot"… forever, in every domain, in every language. That approach doesn't scale, and it's exactly what embeddings retired. We need a way for the computer to compare meanings, not spellings.
The idea: put meaning on a map
Imagine a giant map where every piece of text gets a position, placed so that things that mean similar things land near each other. Leave-related sentences cluster in one region, finance in another, IT support in a third.
Now "find text relevant to this question" becomes "find points near this point" — a distance calculation a computer does millions of times a second. That's the whole trick. Real maps have hundreds or thousands of dimensions instead of two, but nothing else about the intuition changes.
Now the technical name
An embedding is that position: a vector — a fixed-length list of numbers (384, 768, 1536 of them, depending on the model) — produced by an embedding model that was trained so that similar meanings get nearby vectors. Feed it any text, get back its coordinates:
from openai import OpenAI
client = OpenAI()
def embed(text: str) -> list[float]:
r = client.embeddings.create(model="text-embedding-3-small", input=text)
return r.data[0].embedding
v = embed("Employees get 12 casual leaves per calendar year.")
print(len(v), v[:5])Result
1536 [-0.0121, 0.0338, -0.0074, 0.0192, -0.0455]
Those numbers mean nothing individually — nobody can tell you what dimension 41 represents. What matters is only their relationship to other vectors from the same model. Embeddings are meaningful relatively, never absolutely, and that one sentence explains most of the rules in this lesson.
Measuring closeness — cosine similarity
"Near" needs a number. The standard is cosine similarity: it measures the angle between two vectors — same direction ≈ 1, unrelated ≈ 0, opposite ≈ −1. Why angle and not straight-line distance? Because length tends to track how much text there is, and a one-line and a one-paragraph statement of the same idea should still count as similar. Ignore magnitude, keep direction.
import numpy as np
def cosine(a, b):
a, b = np.array(a), np.array(b)
return float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
q = embed("how many CLs do I get?")
d1 = embed("Employees get 12 casual leaves per calendar year.")
d2 = embed("The cafeteria serves lunch from 12:00 to 14:00.")
print(round(cosine(q, d1), 3), round(cosine(q, d2), 3))Result
0.71 0.09
No shared vocabulary with d1 — and it still wins by a mile.
That gap between 0.71 and 0.09 is retrieval working. And a useful practical note: absolute values vary by model, so don't memorize "0.7 means relevant" — what matters is the ranking, and any threshold must be tuned per model on your own data.
Choosing an embedding model
Four things decide it, and cost is rarely the interesting one. Dimension — more dimensions can encode finer distinctions but multiply storage and search cost (a million vectors at 1536 dims ≈ 6 GB in float32, versus ~1.5 GB at 384). Language coverage — if your users write Hinglish, a model trained mostly on English will disappoint; test on real text. Max input length — must comfortably fit your chunk size, or text is silently truncated. Quality on YOUR data — the only criterion that actually matters, and the only one a leaderboard can't tell you. Build a tiny labelled set (query → the chunk that should win) and measure; you'll do exactly that in lesson 9.
Wait — my retrieval got worse and nothing errored
Here's the trap that has cost real teams real weeks. Each embedding model builds its own coordinate system. Dimension seven in model A has nothing to do with dimension seven in model B. So if you index documents with one model and embed queries with another — or upgrade the model and don't re-embed the corpus — every similarity score becomes meaningless noise.
And it does not crash. Vectors of matching size compare happily; the numbers come back; results are quietly garbage. One embedding model for documents and queries — and changing it means re-embedding the entire corpus.
Selection-round radar: "What happens if you change the embedding model but don't reindex?" is a favourite because it separates people who have run a pipeline from people who have read about one. Say: no error, silent quality collapse, incomparable vector spaces, full re-embed required.
What embeddings are genuinely bad at
Embeddings encode meaning, so anything without much meaning to learn embeds poorly. Internal codes like HRA-104, SKUs, invoice numbers, error codes: near-arbitrary tokens whose vectors don't reliably capture their exact identity. Ask for "circular RBI/2024/17" and semantic search may cheerfully return circular 2023/09 — same shape, same meaning-neighbourhood, wrong document.
They're also weak on negation ("leave policy" and "when leave is not granted" sit close together) and on numeric comparison("under ₹5,000" is a filter, not a meaning). Each of these has an engineering answer — keyword/hybrid search and metadata filters — which is precisely the next lesson.
Common mistakes
- Mixing embedding models between indexing and querying — silent, total quality collapse.
- Hard-coding a similarity threshold copied from a blog post — thresholds are model- and data-specific.
- Choosing a model by leaderboard rank without testing on your own domain and language.
- Embedding text longer than the model's limit and not noticing the truncation.
- Expecting embeddings to match exact identifiers — that's keyword search's job.
- Storing vectors without the original text — you'd have nothing to put in the prompt.
Quick recap
| Concept | One-liner |
|---|---|
| Embedding | a vector position on a map of meaning |
| Embedding model | text in → vector out; defines its own coordinate system |
| Cosine similarity | angle between vectors — ignores length, compares meaning |
| Dimension | richness vs storage/search cost; 384–1536 typical |
| The golden rule | same model both sides; change it → re-embed everything |
| Weak spots | exact codes, negation, numeric filters → use hybrid + filters |
Practice Zone — PYQs from real selection rounds
Six MCQs and two hands-on tasks — compute similarities yourself and pick an embedding model like an engineer, not a blog reader.
Two sentences with no words in common get embeddings with high cosine similarity. What does that tell you?
Asked in
Why is cosine similarity the usual choice for comparing embeddings, rather than plain Euclidean distance?
Asked in
You switch to a better embedding model for new documents but leave the old vectors in place. What happens?
Asked in
What does the 'dimension' of an embedding model (e.g. 384 vs 1536) affect most directly?
Asked in
Your RAG bot fails on queries containing internal product codes like 'HRA-104', even though documents mention them. Why do embeddings struggle here?
Asked in
Which is the correct way to embed a user query in a RAG system?
Asked in
Hands-on tasks:
Given three toy vectors, compute cosine similarity and rank which document best matches the query. Predict the ranking before checking.
Asked in
import numpy as np
query = np.array([0.8, 0.5, 0.1]) # "leave policy for new joiners"
docs = {
"probation leave rules": np.array([0.75, 0.55, 0.05]),
"cafeteria timings": np.array([0.05, 0.2, 0.95]),
"annual leave carryover": np.array([0.6, 0.6, 0.2]),
}You're indexing 2 million support tickets in English and Hinglish, searched by agents in real time on a modest budget. What do you check before choosing an embedding model? Name four criteria.
Asked in
FAQ
Are embeddings the same thing the LLM uses internally?
Related but not interchangeable. LLMs embed tokens internally as part of generation; RAG uses a dedicated embedding model that produces one vector for a whole passage. You cannot ask a chat model to "give you the embedding" of a chunk — you call an embeddings endpoint.
Can I embed images, code or tables?
Yes — code embeds fine with general or code-specific models, images need multimodal embedding models, and tables usually work best serialized to readable text (headers included) before embedding.
Is cosine similarity always the right metric?
It's the default and almost always fine. With normalized vectors, cosine and Euclidean produce identical rankings; dot product is used when models are trained for it. Follow your embedding model's documentation rather than a habit.
How much does embedding a large corpus cost?
Far less than people expect — embedding models are tiny compared to chat models, and it's a one-off per document (plus re-embeds on change). The recurring cost in RAG is generation tokens per query, not indexing (lesson 10).
Next lesson: turning similarity into an actual search engine — Lesson 3: Semantic Search vs Keyword Search →