LLMs and Generative AI βΊ Embeddings and Vector Search βΊ Day 264
Day 264: Hybrid Search and Rerankers
Master the dual-stage retrieval paradigm for production RAG systems: combine Lexical Sparse Keyword Search (BM25) with Dense Semantic Vector Search via Reciprocal Rank Fusion (RRF) and convex linear alpha score fusion; integrate Cross-Encoder neural re-rankers (Cohere Rerank, BGE-Reranker-Large, ColBERT late interaction) to evaluate query-document relevance; analyze latency-accuracy trade-offs, and implement an end-to-end Hybrid Search & Re-Ranking pipeline in Python.
Hands-on lab for this lesson
Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/llms-and-generative-ai/day-264-hybrid-search-and-rerankers
- Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git cd ai-roadmap-365.github.io - Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path β section / subsection / week / day:
cd labs/sections/llms-and-generative-ai/day-264-hybrid-search-and-rerankers - Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work β read it before running anything.
- Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
bash tests/run_tests.sh # or the test command named in the lab README
You can also open the lab as a local page (works offline, shows the file tree and expected output).
Learning objectives
By the end of this lesson you will be able to:
- Differentiate between the recall failure modes of pure dense vector search and pure lexical keyword search.
- Implement Reciprocal Rank Fusion (RRF) to merge heterogeneous candidate rankings without score normalization.
- Explain the attention mechanism differences between Bi-Encoders and Cross-Encoders.
- Deploy neural re-rankers (BGE-Reranker, Cohere Rerank) to filter Top-50 retrieval candidates down to Top-5 high-precision passages.
- Build a complete two-stage Hybrid Search and Re-Ranking engine in Python.
Prerequisites
Why this matters
In production Retrieval-Augmented Generation (RAG) systems, relying exclusively on dense vector similarity search creates severe, predictable blind spots:
- When a user searches for an exact alphanumeric product code, SKU, or error code (e.g.
ERR_504_GW_TIMEOUT), dense embedding models map the query to generic βnetwork errorsβ, completely missing the exact technical manual page. - Conversely, when a user searches with conversational synonyms (e.g. βhow do I fix sluggish page loads?β), pure keyword search (BM25) returns zero results if the manual uses the term βlatency optimizationβ.
Hybrid Search & Re-Ranking is the gold standard architecture that bridges this divide. By combining Sparse Keyword Search (BM25) with Dense Semantic Vector Search (HNSW) via Reciprocal Rank Fusion (RRF), and passing the merged candidates through a Cross-Encoder Neural Re-Ranker, you achieve near-perfect retrieval recall and precision across both exact keyword and abstract semantic queries.
Today, you will master the mechanics of BM25 term weighting, Reciprocal Rank Fusion, Cross-Encoder pairwise cross-attention, and ColBERT late interaction, and build an end-to-end Hybrid Search & Re-Ranking engine in Python.
The idea in plain language
Think of hiring a senior software architect from a candidate pool of 10,000 applicants:
- Stage 1A (Lexical Filter - BM25): An automated keyword scanner filters for resumes explicitly mentioning
Kubernetes,Rust, andPostgreSQL(50 candidates). - Stage 1B (Semantic Filter - Vector Search): An AI recruiter matches candidates whose career descriptions reflect βdistributed systems leadershipβ (50 candidates).
- Reciprocal Rank Fusion (RRF): You merge both candidate lists, prioritizing applicants who ranked high in both filters (Deduplicated Top-40 pool).
- Stage 2 (Re-Ranker - Cross-Encoder): The CTO personally conducts a deep 1-on-1 technical interview with the Top-40 candidates, analyzing their exact capabilities to select the Top-3 finalists (High-Precision Re-Ranking).
Historical background
- 1994 (Okapi BM25): Stephen Robertson and Karen SpΓ€rck Jones introduced BM25, establishing the definitive probabilistic term-frequency ranking standard.
- 2009 (Reciprocal Rank Fusion - RRF): Cormack et al. proved that summing inverse rank positions outperforms complex score calibration across heterogeneous search engines.
- 2019 (BERT Cross-Encoders): Nogueira and Cho demonstrated that passing
[Query, Passage]pairs through BERT with full cross-attention yields dramatic gains over dual encoders. - 2020 (ColBERT Late Interaction): Khattab and Zaharia introduced token-level MaxSim late interaction, achieving Cross-Encoder quality at Bi-Encoder speeds.
- 2023β2024 (Modern RAG Standards): Commercial engines (Qdrant, Cohere, Pinecone) integrated hybrid sparse-dense fusion and dedicated neural re-ranking endpoints.
What it is β and what it is not
What Hybrid Search & Re-Ranking IS:
- A Two-Stage Retrieval Pipeline: A system combining fast high-recall candidate generation (Sparse BM25 + Dense Vectors) with a computationally intensive high-precision re-ranking stage (Cross-Encoder).
What it is NOT:
- Not Just Vector Search: Pure vector search ignores exact keyword tokens. Hybrid search explicitly balances lexical and semantic signals.
Why it was created and what problems it solves
Hybrid search and re-ranking solve 4 critical enterprise retrieval challenges:
- The Out-of-Vocabulary / SKU Problem: Catches exact alphanumeric identifiers, error codes, and entity names that embeddings fail to resolve.
- The Synonym & Paraphrase Problem: Recovers documents discussing identical concepts using different vocabulary.
- Score Incompatibility: Reciprocal Rank Fusion merges disparate scoring regimes (BM25 floats from 0 to 50 vs. Cosine similarities from 0 to 1) without calibration errors.
- Elimination of False Positives: Neural cross-encoders weed out passages that share semantic keywords but fail to answer the userβs specific question.
How it works
Let us dissect the mathematical formulation of BM25, RRF rank aggregation, and Cross-Encoder attention mechanics.
1. BM25 (Best Matching 25) Sparse Lexical Scoring
BM25 calculates the relevance score of document D for query Q using term frequencies and inverse document frequencies:
\text{Score}(D, Q) = \sum_{i=1}^{n} \text{IDF}(q_i) \cdot \frac{f(q_i, D) \cdot (k_1 + 1)}{f(q_i, D) + k_1 \cdot <=ft(1 - b + b \cdot \frac{|D|}{\text{avgdl}}\right)}
f(q_i, D): Frequency of termq_iin documentD.|D| / \text{avgdl}: Document length normalized by average corpus document length.k_1(typically 1.2 to 2.0): Controls term frequency saturation.b(typically 0.75): Controls document length penalization.
2. Reciprocal Rank Fusion (RRF) Formulation
When merging candidate lists from BM25 and Dense Vector search:
\text{RRF\_Score}(d) = \sum_{m in M} \frac{1}{k + r_m(d)}
M: Set of retrieval systems (e.g.\{ \text{BM25}, \text{Dense} \}).r_m(d): Rank position of documentdin systemm(1-indexed: 1, 2, 3, β¦).k(typically 60): Smoothing constant preventing top-ranked items from dominating disproportionately.
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β RECIPROCAL RANK FUSION EXAMPLE (k=60) β
βββββββββββββ¬βββββββββββββββ¬ββββββββββββββββ¬ββββββββββββββββββββββββββββββ€
β Document β BM25 Rank β Dense Rank β RRF Calculation β
βββββββββββββΌβββββββββββββββΌββββββββββββββββΌββββββββββββββββββββββββββββββ€
β Doc A β Rank 1 β Rank 2 β 1/(60+1) + 1/(60+2) = 0.0325β
β Doc B β Rank 2 β Not in Top-50 β 1/(60+2) + 0 = 0.0161β
β Doc C β Rank 15 β Rank 1 β 1/(60+15)+ 1/(60+1) = 0.0297β
βββββββββββββ΄βββββββββββββββ΄ββββββββββββββββ΄ββββββββββββββββββββββββββββββ
3. Bi-Encoder vs. Cross-Encoder Architecture
- Bi-Encoder (Dual Encoder):
- Encodes Query
u = E(Q)and Documentv = E(D)independently. - Relevance score:
s = \cos(u, v) = u \cdot v. - Limitation: Query tokens cannot attend to document tokens during encoding.
- Encodes Query
- Cross-Encoder:
- Concatenates Query and Document into a single sequence:
[\text{CLS}] Q [\text{SEP}] D [\text{SEP}]. - Passes the pair through all transformer self-attention layers.
- Every token in the query directly computes cross-attention weights against every token in the document.
- Produces a single scalar relevance probability:
s = \sigma(W \cdot h_{\text{CLS}}).
- Concatenates Query and Document into a single sequence:
4. ColBERT Late Interaction (MaxSim Operator)
ColBERT provides a middle ground:
- Embeds query tokens
\{q_1, ..., q_l\}into a matrix of token vectors. - Embeds document tokens
\{d_1, ..., d_m\}into a matrix of token vectors. - Computes the MaxSim operator: For every query token, finds the maximum cosine similarity among all document tokens, then sums the maximums:
S(Q, D) = \sum_{i=1}^{l} \max_{j=1}^{m} <=ft( e_{q_i} \cdot e_{d_j} \right)
5. Linear Convex Alpha Fusion
An alternative to RRF when calibrated scores are available:
\text{Score}_{\text{final}} = \alpha \cdot \text{Dense\_Norm} + (1 - \alpha) \cdot \text{BM25\_Norm}
\alpha = 0.7: 70% weight on semantic vectors, 30% weight on keyword terms.- Requires min-max score normalization:
s_{\text{norm}} = \frac{s - s_{\min}}{s_{\max} - s_{\min}}.
6. Latency and Throughput Budgeting
In production architectures:
- Stage 1 (Dual Retrieval): BM25 (2ms) + Dense HNSW (3ms) executed in parallel async tasks.
- Stage 2 (RRF Merge): In-memory set union and sorting (0.5ms).
- Stage 3 (Cross-Encoder): Re-ranking Top-50 candidates on GPU/ONNX (12ms).
- Total Retrieval Latency: ~18ms, well within standard 100ms API SLAs.
7. Re-Ranker Model Selection
Popular production re-rankers:
- BGE-Reranker-Large (BAAI): Open-source, 560M parameters, top MTEB re-ranking benchmarks.
- Cohere Rerank v3: Managed API, multilingual, optimized for enterprise documents and JSON schemas.
- FlashRank / MiniLM: Ultra-fast CPU-optimized ONNX re-rankers (sub-5ms on CPU).
8. Evaluating Hybrid Retrieval: MRR@10 and NDCG@10
Measure retrieval quality improvements using standard Information Retrieval metrics:
- Mean Reciprocal Rank (MRR@10): Average reciprocal rank of the first relevant document.
- Normalized Discounted Cumulative Gain (NDCG@10): Evaluates multi-graded relevance across top results.
- Empirical Gain: Adding a cross-encoder typically boosts NDCG@10 by +15% to +25% over standalone vector search.
9. Context Compression & Re-Rank Pruning
After re-ranking:
- Discard candidate passages scoring below a relevance threshold (e.g.
score < 0.2). - Feed only the Top-3 to Top-5 highest-scoring passages to the LLM, reducing prompt token costs and eliminating irrelevant distractor context.
10. Summary: The Two-Stage Gold Standard
- Two-stage hybrid search combines high-recall candidate generation with high-precision cross-encoder re-ranking.
- Reciprocal Rank Fusion seamlessly merges heterogeneous sparse and dense rankings without brittle score scaling.
- In summary, hybrid search and re-ranking deliver the highest retrieval accuracy achievable in modern production RAG systems.
Deep Dive: Mathematical Formulation of Reciprocal Rank Fusion
Reciprocal Rank Fusion (RRF) is an unsupervised rank aggregation method specifically developed to combine ranked candidate lists returned by disparate information retrieval systems (such as lexical inverted indexes and dense vector similarity search) without requiring score normalization or calibration.
Given a set of retrieval systems M (e.g., BM25 and Dense HNSW) and a collection of candidate documents D retrieved across all systems, the RRF score for document d in D is formally defined as:
RRF(d) = sum_(m in M) \frac(1)(k + r_m(d))
Where:
Mis the set of retrieval models (e.g.M = \(\text(BM25), \text(Dense)\)).r_m(d)is the 1-based rank position of documentdwithin the results returned by modelm. If documentdwas not retrieved by systemm,r_m(d) = infinity, yielding an additive score contribution of zero.kis a smoothing ranking constant (typically set empirically tok = 60).
The smoothing constant k plays a crucial role in balancing the influence of high-ranking documents against low-ranking documents:
- Low
kvalues (e.g.,k=10): Strongly prioritize the very top results from each list (1/11 \approx 0.091vs1/20 = 0.050), making the top 1β3 ranked documents dominate the fused scoring. - High
kvalues (e.g.,k=100): Smooth out rank differences, allowing consensus across multiple ranking lists to outweigh an isolated top rank from a single system. - Standard
k=60: Provides optimal empirical performance across standard TREC, BEIR, and MS MARCO benchmarks, ensuring that documents appearing in the top 10 of both sparse and dense retrieval systems receive substantially higher combined scores than documents appearing only in one system.
Comparative Latency Profiling and Hardware Acceleration
When deploying hybrid search pipelines in production enterprise clusters, latency budgeting must be strictly partitioned across sparse indexing, dense approximate nearest neighbor search, and neural re-ranking:
- Sparse Lexical Retrieval (BM25): Operating over inverted memory-mapped postings lists (using Lucene or Tantivy), sparse keyword evaluation executes within 2β5ms on standard multi-core x86 CPUs.
- Dense Approximate Nearest Neighbor (ANN) Search: Vector similarity search using Hierarchical Navigable Small World (HNSW) graphs in Qdrant or Milvus requires 4β8ms for candidate sets of size
K=50. - Cross-Encoder Neural Re-Ranking: Evaluating pairwise token interactions using transformer architectures (e.g., BGE-Reranker-Large or Cohere Rerank v3) is computationally demanding. Scoring 50 candidates sequentially on a single CPU core would require 150β250ms, violating user-interactive SLAs.
- ONNX Runtime and TensorRT Optimization: By quantizing cross-encoder weights to INT8 and compiling execution graphs via ONNX Runtime with AVX-512 VNNI instructions or GPU Tensor Cores, batch scoring latency of 50 candidates is compressed down to 12β18ms.
An everyday analogy
Think of casting actors for a Broadway musical:
- Stage 1 (Open Audition / Hybrid Search):
- Casting Director A scouts for dancers who can hit exact high notes (Keyword BM25).
- Casting Director B scouts for actors with incredible charismatic stage presence (Semantic Vector Search).
- Both directors merge their Top-50 lists into a callback pool (RRF Fusion).
- Stage 2 (Final Callback / Cross-Encoder):
- The Director and Lead Actor run full chemistry reads and duets with each candidate to choose the final lead actor (Deep Cross-Attention).
Examples in practice
Let us build an End-to-End Hybrid Search & RRF Re-Ranking Pipeline:
import math
from collections import Counter
from typing import List, Dict, Any, Tuple
class HybridSearchEngine:
def __init__(self, rrf_k: int = 60):
self.rrf_k = rrf_k
self.documents: List[Dict[str, Any]] = []
self.doc_term_freqs: List[Counter] = []
self.doc_lengths: List[int] = []
self.avg_doc_len: float = 0.0
self.idf: Dict[str, float] = {}
def index_documents(self, docs: List[Dict[str, Any]]):
self.documents = docs
self.doc_term_freqs = []
self.doc_lengths = []
df: Counter = Counter()
for doc in docs:
tokens = doc["text"].lower().split()
tf = Counter(tokens)
self.doc_term_freqs.append(tf)
self.doc_lengths.append(len(tokens))
for term in tf.keys():
df[term] += 1
n_docs = len(docs)
self.avg_doc_len = sum(self.doc_lengths) / max(1, n_docs)
self.idf = {term: math.log((n_docs - count + 0.5) / (count + 0.5) + 1.0) for term, count in df.items()}
def bm25_search(self, query: str, top_k: int = 10, k1: float = 1.5, b: float = 0.75) -> List[Tuple[int, float]]:
q_tokens = query.lower().split()
scores = []
for idx, tf in enumerate(self.doc_term_freqs):
score = 0.0
d_len = self.doc_lengths[idx]
for term in q_tokens:
if term in tf:
term_idf = self.idf.get(term, 0.0)
freq = tf[term]
numerator = freq * (k1 + 1.0)
denominator = freq + k1 * (1.0 - b + b * (d_len / self.avg_doc_len))
score += term_idf * (numerator / denominator)
scores.append((idx, score))
scores.sort(key=lambda x: x[1], reverse=True)
return scores[:top_k]
def reciprocal_rank_fusion(self, sparse_results: List[Tuple[int, float]], dense_results: List[Tuple[int, float]], top_k: int = 5) -> List[Tuple[Dict[str, Any], float]]:
rrf_scores: Dict[int, float] = Counter()
for rank, (doc_idx, _) in enumerate(sparse_results):
rrf_scores[doc_idx] += 1.0 / (self.rrf_k + (rank + 1))
for rank, (doc_idx, _) in enumerate(dense_results):
rrf_scores[doc_idx] += 1.0 / (self.rrf_k + (rank + 1))
ranked_items = sorted(rrf_scores.items(), key=lambda x: x[1], reverse=True)
return [(self.documents[doc_idx], score) for doc_idx, score in ranked_items[:top_k]]
Implications: security, privacy, performance, scalability, and cost
- Re-Ranking Latency Budget: Re-ranking 50 documents takes ~15ms on GPU, but ~150ms on a single CPU core. Use lightweight quantized ONNX models (e.g. FlashRank) when deploying on CPU-only container environments.
- Data Leakage in Re-Ranking: Ensure payload access control filters are applied during Stage 1 retrieval so that private tenant passages are never passed to the re-ranker model.
Alternatives: free, open source, and commercial
| Solution | Type | Primary Strengths |
|---|---|---|
| Qdrant Hybrid | Built-in Sparse + Dense | Single engine, integrated RRF and payload filters |
| Cohere Rerank v3 | Managed Cloud API | Best-in-class relevance, multilingual, zero infra |
| BGE-Reranker | Open Source Transformer | State-of-the-art open weights for self-hosting |
| FlashRank | Lightweight Local ONNX | Ultra-fast sub-5ms CPU re-ranking |
Comparison with related concepts
ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
β SEARCH PARADIGM COMPARISON MATRIX β
ββββββββββββββββββββββ¬ββββββββββββββββββ¬ββββββββββββββββββ¬ββββββββββββββββ€
β Metric β Sparse BM25 β Dense Vector β Hybrid + RRF β
ββββββββββββββββββββββΌββββββββββββββββββΌββββββββββββββββββΌββββββββββββββββ€
β Keyword Accuracy β 100% Exact β Poor on SKUs β 100% Exact β
β Semantic Recall β Poor on Synonymsβ Excellent β Excellent β
β Query Latency β 1 - 3ms β 2 - 5ms β 4 - 8ms β
β Re-Rank Precision β Baseline β Moderate β +20% Accuracy β
ββββββββββββββββββββββ΄ββββββββββββββββββ΄ββββββββββββββββββ΄ββββββββββββββββ
When to use it β and when not to
When to Use Hybrid Search & Re-Ranking:
- Production enterprise RAG, search engines indexing mixed technical/alphanumeric content, customer support bots, and e-commerce product catalogs.
When NOT to use:
- Simple vector prototypes with under 5,000 documents where latency constraints are under 5ms and standard embeddings provide sufficient recall.
The AI connection
- Combining lexical and semantic retrieval beats either alone, reliably, which is why it is the production default rather than an optimisation.
- Reranking is the highest-return addition to a retrieval system β a cross-encoder over the top fifty results improves quality more than most model upgrades.
- The two-stage pattern is everywhere in retrieval: retrieve broadly and cheaply, then rank precisely and expensively.
- Fusion by rank rather than by score avoids having to calibrate two incompatible scoring scales against each other.
- Retrieval quality caps everything downstream, so effort here pays more than the same effort spent on prompting.
Knowledge check
- Why does pure semantic vector search struggle with rare technical terms and alphanumeric codes?
- What is the formula for Reciprocal Rank Fusion (RRF) and why is the smoothing constant
k=60used? - How does the attention mechanism of a Cross-Encoder differ from a Bi-Encoder?
- How does ColBERT late interaction achieve near-Cross-Encoder accuracy at Bi-Encoder speeds?
- What are the typical latency stages of a two-stage hybrid retrieval pipeline?
Hands-on exercise
In this lab, you will build and test an End-to-End Hybrid Search & Re-Ranking Pipeline in Python: implement BM25 sparse keyword scoring, vector cosine scoring, Reciprocal Rank Fusion (RRF) rank aggregation, and a simulated cross-encoder re-ranking filter.
Expected output
[Hybrid Search & Re-Ranking Verification Suite]
Testing BM25 Keyword Search:
Query: 'error 504 gateway' -> Match Doc 0 (Score: 2.14) [PASS]
Testing Dense Semantic Search:
Query: 'sluggish server response' -> Match Doc 1 (Score: 0.92) [PASS]
Testing Reciprocal Rank Fusion (RRF):
Merged Sparse + Dense rankings into unified Top-3 pool [PASS]
Top Combined Result: Doc 0 (RRF Score: 0.0325) [PASS]
Testing Cross-Encoder Re-Ranking:
Re-ranked candidates evaluated with full cross-attention [PASS]
Test Suite: 4 passed in 0.42s
Validate your work
Run the automated test runner:
./tests/run_tests.sh
Troubleshooting
- Ensure document tokens are lowercased and stripped of punctuation for BM25 term frequency consistency.
Common mistakes
- Multiplying raw BM25 and Cosine scores: Unnormalized scores distort fusion; always use RRF rank inversion or min-max normalization.
Practice assignment
- Implement Convex Alpha Linear Fusion: Build a linear score merger with min-max normalization and compare NDCG@5 against RRF.
- Integrate a local FlashRank ONNX Re-Ranker to re-score Top-20 retrieved passages.
Extension challenge
Implement a ColBERT-style MaxSim Late Interaction Engine:
- Compute token-level embedding matrices for query and document pairs, evaluate the token-wise maximum similarity sum, and benchmark query latency against exact Cross-Encoder inference.
Quiz
Q1. Why does pure dense semantic vector search often fail on exact alphanumeric queries like error codes or product SKUs (e.g. "ERR_504_GW")?
- Tokenizers shatter rare identifiers into fragments
- Because vector databases do not accept numbers
- Because embeddings only work on English dictionary words
- Because cosine similarity cannot exceed 0.5
Show answer
Answer: A. Tokenizers shatter rare identifiers into fragments
Dense embeddings map rare strings into generalized semantic regions, losing exact character-level keyword specificity.
Q2. How does Reciprocal Rank Fusion (RRF) calculate the combined score for a document appearing in multiple search candidate lists?
- By multiplying raw BM25 scores with cosine similarity
- Sum of 1 / (k + rank) across the rankings
- By averaging character counts
- By selecting only documents from the first ranking
Show answer
Answer: B. Sum of 1 / (k + rank) across the rankings
RRF uses positional rankings rather than incompatible raw score magnitudes to merge disparate result sets.
Q3. What is the primary architectural difference between a Bi-Encoder and a Cross-Encoder?
- A Bi-Encoder is written in C, and a Cross-Encoder is in Python
- A Bi-Encoder requires two GPUs, while a Cross-Encoder requires four
- A bi-encoder embeds separately; a cross-encoder jointly
- There is no difference
Show answer
Answer: C. A bi-encoder embeds separately; a cross-encoder jointly
Cross-Encoders allow every query token to attend to every document token, achieving vastly superior relevance scoring at higher computational cost.
Q4. Why is it computationally impractical to use a Cross-Encoder to search a 10,000,000 document vector database directly from scratch?
- Because PyTorch disables search loops
- Because Cross-Encoders cannot read English
- Because RAM is limited to 1 MB
- Cross-attention over ten million pairs would take minutes
Show answer
Answer: D. Cross-attention over ten million pairs would take minutes
Cross-Encoders are computationally intensive O(N) forward passes, making them suitable only for re-ranking a small Top-50 candidate pool.
Q5. What role does late-interaction retrieval (such as ColBERT) play in the retrieval spectrum?
- It keeps token vectors and does fast MaxSim matching
- It disables LLM generation
- It compresses text into ZIP archives
- It replaces SQL databases
Show answer
Answer: A. It keeps token vectors and does fast MaxSim matching
ColBERT keeps fine-grained token embeddings and computes MaxSim token alignments in sub-15ms.
Glossary
- Hybrid Search
- A retrieval paradigm combining sparse keyword search (BM25) with dense semantic vector search.
- BM25 (Best Matching 25)
- A probabilistic term-matching ranking function based on Term Frequency and Inverse Document Frequency (TF-IDF).
- Reciprocal Rank Fusion (RRF)
- A rank-aggregation algorithm summing inverse rank positions to combine heterogeneous candidate lists without score calibration.
- Bi-Encoder
- A dual-encoder architecture embedding query and passage independently into fixed-dimensional vectors for fast dot product search.
- Cross-Encoder
- A transformer model processing concatenated query-document token pairs simultaneously with full pairwise cross-attention.
- Re-Ranking
- The second stage of retrieval where a computationally intensive model re-scores a small candidate pool (e.g. Top-50) down to Top-5.
- ColBERT (Late Interaction)
- A multi-vector retrieval model computing token-level MaxSim operations between query and document token matrices.
- Alpha Linear Fusion
- A convex weighted score combination: Final_Score = alpha * Dense_Score + (1 - alpha) * Sparse_Score.
Sources and further reading
- Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods β ACM SIGIR 2009 (accessed 2026-08-30)
- ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT β ACM SIGIR 2020 (accessed 2026-08-30)
- BGE Re-Ranker: Large-scale Cross-Encoder Models for Retrieval Enhancement β Beijing Academy of Artificial Intelligence (BAAI) (accessed 2026-08-30)
Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.