LLMs and Generative AI › Embeddings and Vector Search › Day 260
Hands-on lab — Day 260: What Embeddings Are
- ← Back to the Day 260 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/llms-and-generative-ai/day-260-what-embeddings-are/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/what_embeddings_are_lib.py Test
./tests/run_tests.sh File tree
conftest.py examples/test_what_embeddings_are_lib.py examples/what_embeddings_are_lib.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/starter-run.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/test_what_embeddings_are_lib.py starter/what_embeddings_are_lib.py tests/run_tests.sh tests/test_what_embeddings_are_lib.py troubleshooting.md
Lab README
Lab: Day 260 -- What Embeddings Are
Lesson
Day number: 260 of 365. Course: Course06-SS02 (LLMs and Generative AI - Retrieval and Customization). Topic: Dense Embeddings, Vector Spaces, and Similarity Mathematics.
Purpose
Build and test a Pure NumPy Vector Similarity Engine. Implement L2 vector normalization, cosine similarity, Euclidean distance, multi-document semantic ranking, and Matryoshka dimension truncation.
Learning objectives
- Normalize multi-dimensional vectors to unit sphere length.
- Calculate cosine similarity and Euclidean distance between vectors.
- Rank a collection of candidate document vectors against an incoming query vector.
- Implement Matryoshka sub-vector truncation and re-normalization.
Prerequisites
- Day 259 (Building a CLI Assistant).
- Python 3.11+ with NumPy and Pytest.
Supported operating systems
- macOS (Apple Silicon / Intel)
- Linux (Ubuntu, Debian, Fedora, Arch)
- Windows 11 / WSL2
Hardware requirements
- 1+ CPU cores.
- 1 GB RAM.
- 100 MB disk space.
Required software
- Python 3.11 or newer.
- pip package manager.
- virtualenv or venv module.
Free and open-source options
NumPy and Python standard libraries are open source.
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/what_embeddings_are_lib.py: Student scaffold file.examples/what_embeddings_are_lib.py: Complete reference implementation.tests/test_what_embeddings_are_lib.py: Pytest automated validation suite.expected-output/: Verified output logs and baseline values.
How to run
Execute the reference demonstration script:
python3 examples/what_embeddings_are_lib.py
What the commands do
- Normalizes sample vectors.
- Computes cosine similarity and L2 distance.
- Runs unit test assertions.
Expected output
Embeddings Demo Executed. Cosine Sim: 1.0000, L2 Dist: 0.0000
Validation steps
- Verify L2 normalization produces vectors of length 1.0.
- Confirm identical vectors produce cosine similarity 1.0.
- Validate orthogonal vectors produce cosine similarity 0.0.
- Ensure all unit test assertions pass.
Tests
Run the test runner script:
./tests/run_tests.sh
Cleanup
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +
Troubleshooting
- Zero division error: Handle zero-magnitude vectors gracefully in normalization functions.
Security notes
All vector arithmetic runs locally in memory.
Extension exercises
- Implement a K-Means vector clustering algorithm in pure NumPy.
- Build an embedding dimensionality reduction benchmark.
Navigation
- Lesson title: What Embeddings Are
- Day number: 260 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-260-what-embeddings-are
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-260-what-embeddings-arewhen the site is running.
Expected output
FIELDS.md
# Expected Output Fields: Day 260
- `Embeddings Demo`: Status of embedding similarity execution.
examples-run.txt
Embeddings Demo Executed. Cosine Sim: 1.0000, L2 Dist: 0.0000
measured-values.txt
Embeddings Demo: Success
Cosine Similarity: 1.0000
L2 Distance: 0.0000
starter-run.txt
Starter scaffold executed. Ready for student implementation.
test-run.txt
============================= test session starts ==============================
collected 4 items
tests/test_what_embeddings_are_lib.py::test_normalization_and_cosine_similarity PASSED [ 25%]
tests/test_what_embeddings_are_lib.py::test_document_ranking PASSED [ 50%]
tests/test_what_embeddings_are_lib.py::test_euclidean_distance PASSED [ 75%]
tests/test_what_embeddings_are_lib.py::test_matryoshka_truncation PASSED [100%]
============================== 4 passed in 0.38s ===============================
Source files
conftest.py (585 bytes)
"""Make this lab importable when the suite runs.
pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""
import os
import sys
LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
if os.path.isdir(candidate) and candidate not in sys.path:
sys.path.insert(0, candidate)
examples/test_what_embeddings_are_lib.py (1646 bytes)
import pytest
import numpy as np
from examples.what_embeddings_are_lib import (
VectorSimilarityEngine
)
def test_normalization_and_cosine_similarity():
engine = VectorSimilarityEngine(normalize=True)
v1 = np.array([3.0, 4.0])
norm_v1 = engine.l2_normalize(v1)
assert np.isclose(np.linalg.norm(norm_v1), 1.0)
assert np.isclose(norm_v1[0], 0.6)
assert np.isclose(norm_v1[1], 0.8)
# Identical vectors should have cosine similarity 1.0
assert np.isclose(engine.cosine_similarity(v1, v1), 1.0)
# Orthogonal vectors should have cosine similarity 0.0
v_ortho = np.array([-4.0, 3.0])
assert np.isclose(engine.cosine_similarity(v1, v_ortho), 0.0)
def test_document_ranking():
engine = VectorSimilarityEngine(normalize=True)
query = np.array([1.0, 0.0, 0.0])
doc0 = np.array([0.9, 0.1, 0.0]) # High similarity
doc1 = np.array([0.0, 1.0, 0.0]) # Zero similarity (orthogonal)
doc2 = np.array([0.5, 0.5, 0.0]) # Medium similarity
ranked = engine.rank_documents(query, [doc0, doc1, doc2])
assert ranked[0][0] == 0
assert ranked[1][0] == 2
assert ranked[2][0] == 1
def test_euclidean_distance():
engine = VectorSimilarityEngine(normalize=True)
p1 = np.array([0.0, 0.0])
p2 = np.array([3.0, 4.0])
assert np.isclose(engine.euclidean_distance(p1, p2), 5.0)
def test_matryoshka_truncation():
engine = VectorSimilarityEngine(normalize=True)
vec = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
truncated = engine.matryoshka_truncate(vec, target_dim=3)
assert len(truncated) == 3
assert np.isclose(np.linalg.norm(truncated), 1.0)
examples/what_embeddings_are_lib.py (1594 bytes)
import numpy as np
from typing import List, Tuple
class VectorSimilarityEngine:
def __init__(self, normalize: bool = True):
self.normalize = normalize
def l2_normalize(self, v: np.ndarray) -> np.ndarray:
norm = np.linalg.norm(v)
if norm == 0:
return v
return v / norm
def cosine_similarity(self, u: np.ndarray, v: np.ndarray) -> float:
norm_u = self.l2_normalize(u) if self.normalize else u
norm_v = self.l2_normalize(v) if self.normalize else v
return float(np.dot(norm_u, norm_v))
def euclidean_distance(self, u: np.ndarray, v: np.ndarray) -> float:
return float(np.linalg.norm(u - v))
def rank_documents(self, query_vec: np.ndarray, doc_vecs: List[np.ndarray]) -> List[Tuple[int, float]]:
scores = []
for idx, doc_vec in enumerate(doc_vecs):
sim = self.cosine_similarity(query_vec, doc_vec)
scores.append((idx, sim))
scores.sort(key=lambda x: x[1], reverse=True)
return scores
def matryoshka_truncate(self, v: np.ndarray, target_dim: int) -> np.ndarray:
truncated = v[:target_dim]
return self.l2_normalize(truncated)
def run_embeddings_demo():
engine = VectorSimilarityEngine(normalize=True)
v1 = np.array([1.0, 2.0, 3.0])
v2 = np.array([1.0, 2.0, 3.0])
sim = engine.cosine_similarity(v1, v2)
dist = engine.euclidean_distance(v1, v2)
print(f"Embeddings Demo Executed. Cosine Sim: {sim:.4f}, L2 Dist: {dist:.4f}")
return sim, dist
if __name__ == "__main__":
run_embeddings_demo()
metadata.yml (432 bytes)
lesson_id: D260
day: 260
kind: lab
languages:
- python
setup_commands:
- 'pip install -r requirements/requirements.txt'
run_commands:
- 'python3 examples/what_embeddings_are_lib.py'
test_commands:
- './tests/run_tests.sh'
cleanup_commands:
- 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/test_what_embeddings_are_lib.py (1646 bytes)
import pytest
import numpy as np
from examples.what_embeddings_are_lib import (
VectorSimilarityEngine
)
def test_normalization_and_cosine_similarity():
engine = VectorSimilarityEngine(normalize=True)
v1 = np.array([3.0, 4.0])
norm_v1 = engine.l2_normalize(v1)
assert np.isclose(np.linalg.norm(norm_v1), 1.0)
assert np.isclose(norm_v1[0], 0.6)
assert np.isclose(norm_v1[1], 0.8)
# Identical vectors should have cosine similarity 1.0
assert np.isclose(engine.cosine_similarity(v1, v1), 1.0)
# Orthogonal vectors should have cosine similarity 0.0
v_ortho = np.array([-4.0, 3.0])
assert np.isclose(engine.cosine_similarity(v1, v_ortho), 0.0)
def test_document_ranking():
engine = VectorSimilarityEngine(normalize=True)
query = np.array([1.0, 0.0, 0.0])
doc0 = np.array([0.9, 0.1, 0.0]) # High similarity
doc1 = np.array([0.0, 1.0, 0.0]) # Zero similarity (orthogonal)
doc2 = np.array([0.5, 0.5, 0.0]) # Medium similarity
ranked = engine.rank_documents(query, [doc0, doc1, doc2])
assert ranked[0][0] == 0
assert ranked[1][0] == 2
assert ranked[2][0] == 1
def test_euclidean_distance():
engine = VectorSimilarityEngine(normalize=True)
p1 = np.array([0.0, 0.0])
p2 = np.array([3.0, 4.0])
assert np.isclose(engine.euclidean_distance(p1, p2), 5.0)
def test_matryoshka_truncation():
engine = VectorSimilarityEngine(normalize=True)
vec = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
truncated = engine.matryoshka_truncate(vec, target_dim=3)
assert len(truncated) == 3
assert np.isclose(np.linalg.norm(truncated), 1.0)
starter/what_embeddings_are_lib.py (914 bytes)
import numpy as np
from typing import List, Tuple
class VectorSimilarityEngine:
def __init__(self, normalize: bool = True):
# TODO: Initialize vector similarity engine
pass
def l2_normalize(self, v: np.ndarray) -> np.ndarray:
# TODO: Normalize vector to unit length (L2 norm = 1)
pass
def cosine_similarity(self, u: np.ndarray, v: np.ndarray) -> float:
# TODO: Compute cosine similarity
pass
def euclidean_distance(self, u: np.ndarray, v: np.ndarray) -> float:
# TODO: Compute Euclidean distance
pass
def rank_documents(self, query_vec: np.ndarray, doc_vecs: List[np.ndarray]) -> List[Tuple[int, float]]:
# TODO: Rank document vectors by descending similarity
pass
def matryoshka_truncate(self, v: np.ndarray, target_dim: int) -> np.ndarray:
# TODO: Truncate vector and re-normalize
pass
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 260 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_what_embeddings_are_lib.py (1646 bytes)
import pytest
import numpy as np
from examples.what_embeddings_are_lib import (
VectorSimilarityEngine
)
def test_normalization_and_cosine_similarity():
engine = VectorSimilarityEngine(normalize=True)
v1 = np.array([3.0, 4.0])
norm_v1 = engine.l2_normalize(v1)
assert np.isclose(np.linalg.norm(norm_v1), 1.0)
assert np.isclose(norm_v1[0], 0.6)
assert np.isclose(norm_v1[1], 0.8)
# Identical vectors should have cosine similarity 1.0
assert np.isclose(engine.cosine_similarity(v1, v1), 1.0)
# Orthogonal vectors should have cosine similarity 0.0
v_ortho = np.array([-4.0, 3.0])
assert np.isclose(engine.cosine_similarity(v1, v_ortho), 0.0)
def test_document_ranking():
engine = VectorSimilarityEngine(normalize=True)
query = np.array([1.0, 0.0, 0.0])
doc0 = np.array([0.9, 0.1, 0.0]) # High similarity
doc1 = np.array([0.0, 1.0, 0.0]) # Zero similarity (orthogonal)
doc2 = np.array([0.5, 0.5, 0.0]) # Medium similarity
ranked = engine.rank_documents(query, [doc0, doc1, doc2])
assert ranked[0][0] == 0
assert ranked[1][0] == 2
assert ranked[2][0] == 1
def test_euclidean_distance():
engine = VectorSimilarityEngine(normalize=True)
p1 = np.array([0.0, 0.0])
p2 = np.array([3.0, 4.0])
assert np.isclose(engine.euclidean_distance(p1, p2), 5.0)
def test_matryoshka_truncation():
engine = VectorSimilarityEngine(normalize=True)
vec = np.array([1.0, 2.0, 3.0, 4.0, 5.0, 6.0])
truncated = engine.matryoshka_truncate(vec, target_dim=3)
assert len(truncated) == 3
assert np.isclose(np.linalg.norm(truncated), 1.0)
Troubleshooting
Troubleshooting: Day 260 - What Embeddings Are
Common Issues
- Cosine similarity out of range [-1, 1]:
- Cause: Numerical floating-point roundoff error in dot products.
- Fix: Use np.clip(sim, -1.0, 1.0) before downstream arcsin/arccos calculations.
Security notes
Security & Privacy: Day 260 - What Embeddings Are
Security Guidance
- Restrict access to raw vector embeddings of sensitive personal identifiable information (PII) to prevent embedding inversion extraction attacks.