RAG Retrieval Techniques: Hybrid Search, Reranking, Query Rewriting & HyDE

Hiring works in stages: a recruiter skims a thousand resumes in a day (fast, shallow), a shortlist gets a real interview (slow, accurate), and one person gets hired. Nobody interviews a thousand candidates, and nobody hires from a keyword skim. Production retrieval works exactly the same way — and this lesson is where a basic RAG demo turns into a system that actually finds the right chunk.

Why plain top-k stops working

The basic recipe — embed the question, take the three nearest chunks — works beautifully in a demo with 200 chunks. On a real corpus it starts missing, in four recognizable ways:

1 · Vocabulary gaps. Users say "maternity leave"; the handbook says "birth-related entitlement" under "Parental Benefits". 2 · Exact identifiers. "Circular RBI/2024/17" retrieves circular 2023/09 — same shape, same neighbourhood, wrong document. 3 · Conversational questions. "And for interns?" is unsearchable on its own. 4 · Near-misses ranked above the answer. The right chunk is 7th; you only fetched 3.

Each has a specific fix, and together they form the funnel.

Hybrid search — two searches, one ranking

Fixes gap 1 and 2 at once. Run dense (vector) and sparse (BM25) retrieval in parallel: dense catches paraphrases, sparse catches exact rare terms. Then merge — and merging is the interesting part, because cosine similarity lives around 0–1 while BM25 is unbounded and corpus-dependent. Adding them lets whichever scale happens to be bigger dominate.

Reciprocal Rank Fusion (RRF) sidesteps the problem entirely by using only each document's rank:

def rrf(rankings, k=60):
    """Merge ranked lists using ranks only - scales never compared."""
    scores = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

print(rrf([["d3", "d1", "d7"],      # vector hits
           ["d7", "d3", "d9"]]))    # keyword hits

['d3', 'd7', 'd1', 'd9']

The constant k (commonly 60) damps the influence of the very top positions so one confident-but-wrong list can't bulldoze the other. Fuse by rank, never by raw score.

Reranking — the interview after the resume skim

Fixes gap 4. Vector search is a bi-encoder: the query and each chunk were embedded separately, so the comparison never actually reads them together. That's what makes it fast enough for millions of chunks — and also what makes it approximate.

A reranker is a cross-encoder: it feeds the query and a candidate chunk into one model together and scores their relevance directly. Far more accurate, far too slow to run over a whole corpus. Hence the pattern:

candidates = hybrid_search(question, k=40)          # cheap, broad

pairs = [(question, c.text) for c in candidates]
scores = reranker.predict(pairs)                    # cross-encoder

top = [c for _, c in sorted(zip(scores, candidates),
                            key=lambda x: x[0], reverse=True)][:4]

Retrieve wide, rerank narrow. In practice this is the single biggest quality upgrade available to a struggling RAG system — bigger than swapping the LLM — because it fixes the ranking rather than hoping the model reads past the noise. Cost: a few hundred milliseconds.

Query rewriting — make the question searchable

Fixes gap 3. Conversations are full of unsearchable turns:

One cheap LLM call takes the conversation history plus the latest message and produces a standalone query. Multi-query expansion goes further: generate three phrasings, retrieve for each, fuse the results with RRF. More recall, more cost — measure before adopting.

Wait — questions and answers don't even look alike

Here's a subtle problem worth knowing by name. You embed a question and compare it against documents — but questions and statements are written differently. "How do I claim medical bills?" doesn't textually resemble "Claims must be submitted within 90 days using form MR-2", even though one answers the other.

HyDE — Hypothetical Document Embeddings — closes the gap with a lovely trick: ask the LLM to draft an imaginary answer, then search using that text's embedding. The draft may be factually wrong; it doesn't matter. It only has to look like the document you're hunting for. Cost: one extra LLM call before retrieval.

The full funnel

Read it as a sequence of trade-offs, not a stack of buzzwords. Each stage exists because the previous one is fast but blunt: cheap search buys recall, fusion reconciles two views, the reranker buys precision, and only four chunks are expensive enough (in tokens) to deserve the prompt. You don't need all of it on day one — add each stage when measurement shows the failure it fixes, which is why lesson 9 exists.

Selection-round radar: "How would you improve a RAG system whose answers are often irrelevant?" is extremely common. The strong answer walks the funnel: check retrieval first (log the chunks), add hybrid search for vocabulary/identifier gaps, add a reranker for ordering, add query rewriting for conversational turns — and measure recall@k before and after each change.

Tuning k — more is not better

Raising top-k from 3 to 20 feels safe and usually isn't. You get better recall, yes — plus seventeen mediocre chunks crowding the prompt, triple the token cost, and a model whose attention is now split across a haystack. The mature pattern is wide retrieval, narrow context: fetch 30–50 candidates, rerank, keep 3–5.

Also worth setting: a relevance threshold. If even the best reranked chunk scores poorly, the honest move is to retrieve nothing and let the bot say it doesn't know — lesson 7 wires that refusal path in properly.

Common mistakes

• Adding a reranker before checking whether recall@k is the problem — a reranker cannot promote a chunk that was never retrieved.
• Summing cosine and BM25 scores instead of fusing by rank.
• Raising top-k as a substitute for reranking — more noise, more cost.
• Skipping query rewriting in a conversational bot, then blaming the embeddings for follow-up failures.
• Adopting HyDE, multi-query and reranking all at once, so nobody knows which one helped.
• No relevance threshold — the bot always answers, even when the best chunk is junk.

Quick recap

Practice Zone — PYQs from real selection rounds

Six MCQs and two serious tasks — design a bank's retrieval stack with a latency budget, and diagnose a real vocabulary-gap miss.

Q1Retrievalmedium

What does a reranker (cross-encoder) do that the initial vector search cannot?

Asked in

aSearch faster over millions of vectorsbScore each candidate by reading the query and the chunk TOGETHER, which is far more accurate than comparing two independently-made embeddingscCreate the embeddings in the first placedReduce the number of documents in the index

Q2Retrievalmedium

Why is retrieve-then-rerank preferred over just reranking everything?

Asked in

aRerankers only accept 50 documents by designbCross-encoders are far too slow to score millions of chunks per query — the cheap vector search narrows the field so the expensive model only judges a shortlistcReranking would need a bigger vector databasedIt isn't preferred; reranking everything is standard

Q3Retrievalmedium

What is query rewriting/expansion in RAG, and why does it help?

Asked in

aCorrecting the user's grammar before displaybUsing an LLM to turn a vague or conversational question into one or more explicit search queries (resolving 'it', adding synonyms), improving what retrieval can matchcRewriting the retrieved chunksdTranslating the answer into the user's language

Q4Retrievalhard

What is HyDE (Hypothetical Document Embeddings)?

Asked in

aA method that stores hypothetical documents in the indexbThe LLM drafts a hypothetical *answer* to the question, and that answer's embedding is used for retrieval — because an answer looks more like the target document than a question doescA compression scheme for embeddingsdA reranking algorithm

Q5Retrievalmedium

Increasing top-k from 3 to 20 usually causes:

Asked in

aStrictly better answersbBetter recall but more noise, higher token cost per query, and often worse answers unless a reranker trims the listcNo change at alldFaster responses

Q6Retrievalmedium

In hybrid search, why can't you just add the vector score and the BM25 score together?

Asked in

aBecause addition is computationally expensivebThey're on completely different, unbounded scales — so scores must be normalized, weighted, or fused by rank (e.g. RRF) insteadcBecause BM25 scores are always negativedYou can, and it always works well

Hands-on tasks:

FAQ

Next lesson: put all of it together in ~30 lines of Python — Lesson 7: Build a RAG Pipeline →