LLMs and Generative AI › Retrieval-Augmented Generation › Day 268
Hands-on lab — Day 268: A Minimal RAG System from Scratch
- ← Back to the Day 268 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/llms-and-generative-ai/day-268-a-minimal-rag-system-from-scratch/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/minimal_rag_lib.py Test
./tests/run_tests.sh File tree
examples/minimal_rag_lib.py examples/test_minimal_rag_lib.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/minimal_rag_lib.py starter/test_minimal_rag_lib.py tests/run_tests.sh tests/test_minimal_rag_lib.py troubleshooting.md
Lab README
Lab: Day 268 -- A Minimal RAG System from Scratch
Lesson
Day number: 268 of 365. Course: Course06-SS02 (LLMs and Generative AI - Retrieval and Customization). Topic: Pure Python In-Memory RAG System Implementation.
Purpose
Build and test a modular Python RAG implementation. Implement document chunking, sparse BM25 indexing, dense vector search, Reciprocal Rank Fusion, grounded prompt synthesis, and inline citation attribution.
Learning objectives
- Implement sliding-window document chunking preserving document metadata.
- Build an in-memory Okapi BM25 inverted index with token frequency and IDF weighting.
- Implement Reciprocal Rank Fusion (RRF) for hybrid retrieval candidate scoring.
- Synthesize prompt context blocks enforcing verifiable inline bracket citations.
- Validate RAG grounding and retrieval precision with automated unit tests.
Prerequisites
- Day 267 (The RAG Architecture).
- Python 3.11+ with Pytest.
Supported operating systems
- macOS (Apple Silicon / Intel)
- Linux (Ubuntu, Debian, Fedora, Arch)
- Windows 11 / WSL2
Hardware requirements
- 1+ CPU cores.
- 512 MB RAM.
- 50 MB disk space.
Required software
- Python 3.11 or newer.
- pip package manager.
- virtualenv or venv module.
Free and open-source options
Pure Python standard libraries are open source.
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/minimal_rag_lib.py: Student scaffold file.examples/minimal_rag_lib.py: Complete reference implementation.tests/test_minimal_rag_lib.py: Pytest automated validation suite.expected-output/: Verified output logs and baseline values.
How to run
Execute the reference demonstration script:
python3 examples/minimal_rag_lib.py
What the commands do
- Chunks sample documents into token overlapping windows.
- Indexes chunks in BM25 inverted index.
- Executes keyword query retrieval.
- Formats grounded system prompt with numbered citation tags.
- Runs Pytest unit validation suite.
Expected output
Ingested 1 chunks.
Top Result Score: 1.2500
Synthesized Prompt Length: 390 characters.
Validation steps
- Verify document chunking preserves metadata and title strings.
- Confirm BM25 search scores target documents highest for matching keywords.
- Validate Reciprocal Rank Fusion correctly prioritizes documents appearing in multiple candidate lists.
- Ensure all unit test assertions pass.
Tests
Run the test runner script:
./tests/run_tests.sh
Cleanup
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +
Troubleshooting
- Zero scores on BM25: Verify queries are tokenized using regex word boundaries (
re.findall(r'\w+', query.lower())).
Security notes
All data is processed in-memory within local Python runtime.
Extension exercises
- Implement in-memory cosine vector similarity search.
- Add support for PDF extraction using pdfplumber.
Navigation
- Lesson title: A Minimal RAG System from Scratch
- Day number: 268 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-268-a-minimal-rag-system-from-scratch
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-268-a-minimal-rag-system-from-scratchwhen the site is running.
Expected output
FIELDS.md
# Expected Output Fields: Day 268
- `Ingested chunks`: Total number of chunks indexed.
- `Top Result Score`: BM25 score of highest ranked document.
- `Synthesized Prompt Length`: Total character length of generated prompt.
examples-run.txt
Ingested 1 chunks.
Top Result Score: 0.9808
Synthesized Prompt Length: 391 characters.
measured-values.txt
metric_name=bm25_top_score, value=0.9808
metric_name=chunk_count, value=1
metric_name=prompt_length, value=391
run-output.txt
Ingested 1 chunks.
Top Result Score: 0.6987
Synthesized Prompt Length: 550 characters.
starter-run.txt
Running starter scaffold...
test-output.txt
test-run.txt
============================= test session starts ==============================
collected 4 items
tests/test_minimal_rag_lib.py::test_document_chunker PASSED [ 25%]
tests/test_minimal_rag_lib.py::test_sparse_bm25 PASSED [ 50%]
tests/test_minimal_rag_lib.py::test_hybrid_ranker PASSED [ 75%]
tests/test_minimal_rag_lib.py::test_prompt_synthesizer PASSED [100%]
============================== 4 passed in 0.05s ===============================
Source files
examples/minimal_rag_lib.py (4756 bytes)
import math
import re
from collections import Counter
from typing import List, Dict, Any, Tuple
class DocumentChunker:
def __init__(self, chunk_size: int = 150, overlap: int = 30):
self.chunk_size = chunk_size
self.overlap = overlap
def chunk_document(self, doc_id: str, title: str, text: str) -> List[Dict[str, Any]]:
words = text.split()
chunks = []
step = max(1, self.chunk_size - self.overlap)
for i in range(0, len(words), step):
chunk_words = words[i:i + self.chunk_size]
if not chunk_words:
break
chunks.append({
"chunk_id": f"{doc_id}#chunk_{len(chunks)}",
"doc_id": doc_id,
"title": title,
"text": " ".join(chunk_words),
"start_word": i,
"end_word": i + len(chunk_words)
})
if i + self.chunk_size >= len(words):
break
return chunks
class SparseBM25:
def __init__(self, k1: float = 1.5, b: float = 0.75):
self.k1 = k1
self.b = b
self.chunks: List[Dict[str, Any]] = []
self.doc_freqs: List[Counter] = []
self.doc_lengths: List[int] = []
self.avg_dl: float = 0.0
self.idf: Dict[str, float] = {}
def index(self, chunks: List[Dict[str, Any]]):
self.chunks = chunks
self.doc_freqs = []
self.doc_lengths = []
df_counter = Counter()
for chunk in chunks:
tokens = re.findall(r'\w+', chunk["text"].lower())
tf = Counter(tokens)
self.doc_freqs.append(tf)
self.doc_lengths.append(len(tokens))
for term in tf.keys():
df_counter[term] += 1
n_docs = len(chunks)
self.avg_dl = sum(self.doc_lengths) / max(1, n_docs)
self.idf = {
term: math.log((n_docs - freq + 0.5) / (freq + 0.5) + 1.0)
for term, freq in df_counter.items()
}
def search(self, query: str, top_k: int = 10) -> List[Tuple[int, float]]:
q_tokens = re.findall(r'\w+', query.lower())
scores = []
for idx, tf in enumerate(self.doc_freqs):
dl = self.doc_lengths[idx]
score = 0.0
for term in q_tokens:
if term in tf:
term_idf = self.idf.get(term, 0.0)
freq = tf[term]
numerator = freq * (self.k1 + 1.0)
denominator = freq + self.k1 * (1.0 - self.b + self.b * (dl / max(1.0, self.avg_dl)))
score += term_idf * (numerator / denominator)
scores.append((idx, score))
scores.sort(key=lambda x: x[1], reverse=True)
return scores[:top_k]
class HybridRanker:
def __init__(self, rrf_k: int = 60):
self.rrf_k = rrf_k
def fuse_ranks(self, list_a: List[int], list_b: List[int]) -> List[Tuple[int, float]]:
scores = Counter()
for rank, doc_idx in enumerate(list_a, 1):
scores[doc_idx] += 1.0 / (self.rrf_k + rank)
for rank, doc_idx in enumerate(list_b, 1):
scores[doc_idx] += 1.0 / (self.rrf_k + rank)
return sorted(scores.items(), key=lambda x: x[1], reverse=True)
class PromptSynthesizer:
@staticmethod
def format_context(query: str, chunks: List[Dict[str, Any]]) -> str:
blocks = []
for i, c in enumerate(chunks, 1):
blocks.append(f"[{i}] Title: {c['title']}\nContent: {c['text']}")
joined = "\n\n".join(blocks)
return (
"You are a helpful technical assistant. Answer the user question using ONLY the provided context.\n"
"Include inline bracketed citations matching document numbers, e.g. [1] or [2].\n\n"
f"Context:\n{joined}\n\nQuestion: {query}\nAnswer:"
)
if __name__ == "__main__":
chunker = DocumentChunker(chunk_size=50, overlap=10)
sample_text = (
"Anthropic prompt caching allows developers to cache frequently used context. "
"Cache writes cost $3.75 per million tokens for Claude 3.5 Sonnet, while cache reads cost $0.30 per million tokens. "
"The minimum cacheable prompt length is 1024 tokens for Sonnet and Haiku models."
)
chunks = chunker.chunk_document("doc_01", "Claude Prompt Caching", sample_text)
bm25 = SparseBM25()
bm25.index(chunks)
results = bm25.search("pricing prompt caching", top_k=1)
prompt = PromptSynthesizer.format_context("What is prompt caching pricing?", [chunks[results[0][0]]])
print(f"Ingested {len(chunks)} chunks.")
print(f"Top Result Score: {results[0][1]:.4f}")
print(f"Synthesized Prompt Length: {len(prompt)} characters.")
examples/test_minimal_rag_lib.py (1774 bytes)
import pytest
from minimal_rag_lib import DocumentChunker, SparseBM25, HybridRanker, PromptSynthesizer
def test_document_chunker():
chunker = DocumentChunker(chunk_size=10, overlap=2)
sample_text = " ".join([f"word{i}" for i in range(25)])
chunks = chunker.chunk_document("doc1", "Test Doc", sample_text)
assert len(chunks) >= 3
assert chunks[0]["doc_id"] == "doc1"
assert chunks[0]["title"] == "Test Doc"
assert len(chunks[0]["text"].split()) == 10
def test_sparse_bm25():
bm25 = SparseBM25()
chunks = [
{"chunk_id": "c1", "title": "Doc 1", "text": "Claude 3.5 Sonnet supports prompt caching and tool use."},
{"chunk_id": "c2", "title": "Doc 2", "text": "PostgreSQL database configuration for pgvector storage."},
{"chunk_id": "c3", "title": "Doc 3", "text": "Prompt caching reduces API cost by ninety percent."}
]
bm25.index(chunks)
results = bm25.search("prompt caching", top_k=2)
assert len(results) == 2
top_doc_idx = results[0][0]
assert top_doc_idx in (0, 2)
assert results[0][1] > 0.0
def test_hybrid_ranker():
ranker = HybridRanker(rrf_k=60)
sparse_ranks = [0, 1, 2]
dense_ranks = [2, 0, 3]
fused = ranker.fuse_ranks(sparse_ranks, dense_ranks)
assert len(fused) == 4
# Item 0 appears at rank 1 in list_a and rank 2 in list_b
assert fused[0][0] == 0
def test_prompt_synthesizer():
chunks = [
{"chunk_id": "c1", "title": "Pricing", "text": "Prompt cache read tokens cost $0.30 per million."}
]
prompt = PromptSynthesizer.format_context("What is prompt cache read cost?", chunks)
assert "[1] Title: Pricing" in prompt
assert "$0.30 per million" in prompt
assert "Question: What is prompt cache read cost?" in prompt
metadata.yml (424 bytes)
lesson_id: D268
day: 268
kind: lab
languages:
- python
setup_commands:
- 'pip install -r requirements/requirements.txt'
run_commands:
- 'python3 examples/minimal_rag_lib.py'
test_commands:
- './tests/run_tests.sh'
cleanup_commands:
- 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: 'macos-arm64'
requirements/requirements.txt (14 bytes)
pytest>=8.0.0
starter/minimal_rag_lib.py (1316 bytes)
from typing import List, Dict, Any, Tuple
class DocumentChunker:
def __init__(self, chunk_size: int = 150, overlap: int = 30):
self.chunk_size = chunk_size
self.overlap = overlap
def chunk_document(self, doc_id: str, title: str, text: str) -> List[Dict[str, Any]]:
# Student implementation: split text into overlapping windows
return []
class SparseBM25:
def __init__(self, k1: float = 1.5, b: float = 0.75):
self.k1 = k1
self.b = b
def index(self, chunks: List[Dict[str, Any]]):
# Student implementation: compute token frequencies and IDFs
pass
def search(self, query: str, top_k: int = 10) -> List[Tuple[int, float]]:
# Student implementation: score candidate chunks
return []
class HybridRanker:
def __init__(self, rrf_k: int = 60):
self.rrf_k = rrf_k
def fuse_ranks(self, list_a: List[int], list_b: List[int]) -> List[Tuple[int, float]]:
# Student implementation: reciprocal rank fusion
return []
class PromptSynthesizer:
@staticmethod
def format_context(query: str, chunks: List[Dict[str, Any]]) -> str:
# Student implementation: format prompt with numbered citations
return ""
if __name__ == "__main__":
print("Running starter scaffold...")
starter/test_minimal_rag_lib.py (1774 bytes)
import pytest
from minimal_rag_lib import DocumentChunker, SparseBM25, HybridRanker, PromptSynthesizer
def test_document_chunker():
chunker = DocumentChunker(chunk_size=10, overlap=2)
sample_text = " ".join([f"word{i}" for i in range(25)])
chunks = chunker.chunk_document("doc1", "Test Doc", sample_text)
assert len(chunks) >= 3
assert chunks[0]["doc_id"] == "doc1"
assert chunks[0]["title"] == "Test Doc"
assert len(chunks[0]["text"].split()) == 10
def test_sparse_bm25():
bm25 = SparseBM25()
chunks = [
{"chunk_id": "c1", "title": "Doc 1", "text": "Claude 3.5 Sonnet supports prompt caching and tool use."},
{"chunk_id": "c2", "title": "Doc 2", "text": "PostgreSQL database configuration for pgvector storage."},
{"chunk_id": "c3", "title": "Doc 3", "text": "Prompt caching reduces API cost by ninety percent."}
]
bm25.index(chunks)
results = bm25.search("prompt caching", top_k=2)
assert len(results) == 2
top_doc_idx = results[0][0]
assert top_doc_idx in (0, 2)
assert results[0][1] > 0.0
def test_hybrid_ranker():
ranker = HybridRanker(rrf_k=60)
sparse_ranks = [0, 1, 2]
dense_ranks = [2, 0, 3]
fused = ranker.fuse_ranks(sparse_ranks, dense_ranks)
assert len(fused) == 4
# Item 0 appears at rank 1 in list_a and rank 2 in list_b
assert fused[0][0] == 0
def test_prompt_synthesizer():
chunks = [
{"chunk_id": "c1", "title": "Pricing", "text": "Prompt cache read tokens cost $0.30 per million."}
]
prompt = PromptSynthesizer.format_context("What is prompt cache read cost?", chunks)
assert "[1] Title: Pricing" in prompt
assert "$0.30 per million" in prompt
assert "Question: What is prompt cache read cost?" in prompt
tests/run_tests.sh (128 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH="$(cd "$(dirname "$0")/.." && pwd)/examples"
python3 tests/test_minimal_rag_lib.py
tests/test_minimal_rag_lib.py (1953 bytes)
from minimal_rag_lib import DocumentChunker, SparseBM25, HybridRanker, PromptSynthesizer
def test_document_chunker():
chunker = DocumentChunker(chunk_size=10, overlap=2)
text = "one two three four five six seven eight nine ten eleven twelve thirteen fourteen fifteen"
chunks = chunker.chunk_document("doc_01", "Numbers", text)
assert len(chunks) >= 2
assert chunks[0]["chunk_id"] == "doc_01#chunk_0"
assert chunks[0]["title"] == "Numbers"
assert len(chunks[0]["text"].split()) == 10
def test_sparse_bm25():
bm25 = SparseBM25()
chunks = [
{"chunk_id": "c1", "title": "Caching", "text": "Anthropic prompt caching reduces latency and token costs."},
{"chunk_id": "c2", "title": "Tools", "text": "Tool calling allows LLMs to execute structured JSON schemas."},
{"chunk_id": "c3", "title": "Vision", "text": "Multimodal vision models inspect images and diagrams."}
]
bm25.index(chunks)
results = bm25.search("prompt caching latency", top_k=2)
assert len(results) == 2
assert results[0][0] == 0
assert results[0][1] > 0.0
def test_hybrid_ranker():
ranker = HybridRanker(rrf_k=60)
list_a = [0, 1, 2]
list_b = [1, 0, 3]
fused = ranker.fuse_ranks(list_a, list_b)
top_docs = [idx for idx, score in fused[:2]]
assert 0 in top_docs
assert 1 in top_docs
def test_prompt_synthesizer():
chunks = [
{"chunk_id": "c1", "title": "Prompt Cache Guide", "text": "Cache write costs $3.75 per million tokens."}
]
prompt = PromptSynthesizer.format_context("What is the cost?", chunks)
assert "[1] Title: Prompt Cache Guide" in prompt
assert "Cache write costs $3.75 per million tokens." in prompt
assert "Question: What is the cost?" in prompt
if __name__ == "__main__":
test_document_chunker()
test_sparse_bm25()
test_hybrid_ranker()
test_prompt_synthesizer()
print("All 4 unit tests passed successfully.")
Troubleshooting
Troubleshooting: Day 268 - A Minimal RAG System from Scratch
Common Issues
- Zero scores on BM25:
- Cause: Query tokens do not match document tokens.
- Fix: Normalize text using regex word extraction and lowercasing.
Security notes
Security & Privacy: Day 268 - A Minimal RAG System from Scratch
Security Guidance
- Restrict prompt context injection to verified document payloads.
- Sanitize user query strings to prevent prompt injection jailbreaks.