Sandeep Bazar 365 Days of AI
Labs

Machine Learning › Unsupervised Learning › Day 186

Hands-on lab — Day 186: t-SNE and UMAP

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/t_sne_and_umap_lib.py

Test

./tests/run_tests.sh

File tree

examples/t_sne_and_umap_lib.py
examples/test_t_sne_and_umap_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/t_sne_and_umap_lib.py
starter/test_t_sne_and_umap_lib.py
tests/run_tests.sh
tests/test_t_sne_and_umap_lib.py
troubleshooting.md

Lab README

Lab: Day 186 -- t-SNE and UMAP

Lesson

Day number: 186 of 365. Course: Course04-SS03 (Beyond Supervised Learning). Topic: t-SNE, UMAP, and Non-Linear Manifold Dimensionality Reduction.

Purpose

Build a complete, pure NumPy implementation of t-Distributed Stochastic Neighbor Embedding (t-SNE) from scratch. You will implement high-dimensional Gaussian affinity matrix construction, low-dimensional Student-t probability calculation, analytical KL-divergence gradient computation, and momentum-based coordinate updates.

Learning objectives

  • Calculate pairwise Gaussian probability affinities with adaptive variance.
  • Symmetrize probability distributions and apply early exaggeration.
  • Compute low-dimensional Student-t probabilities to prevent crowding.
  • Execute gradient descent with momentum on coordinate embeddings.

Prerequisites

  • Multivariable calculus: Gradient descent optimization.
  • Probability: Gaussian and Student-t probability density distributions, KL divergence.
  • Python 3.11+ with NumPy.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 512 MB RAM.
  • 50 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

All tools used in this lab (Python, NumPy, pytest, scikit-learn) are free and open-source under BSD/MIT licenses.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/t_sne_and_umap_lib.py: Student scaffold file.
  • examples/t_sne_and_umap_lib.py: Complete reference implementation.
  • tests/test_t_sne_and_umap_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/t_sne_and_umap_lib.py

What the commands do

  • Generates two 4D Gaussian clusters with 80 total samples.
  • Executes TSNEFromScratch to optimize 2D embedding coordinates.
  • Logs output embedding dimensions.

Expected output

t-SNE Demo: Transformed (80, 4) to Embedding (80, 2)

Validation steps

  1. Verify that high-dimensional affinity matrix P is symmetric and sums to 1.0.
  2. Verify that low-dimensional embedding coordinates Y do not contain NaN.
  3. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Exploding Gradients: Reduce learning rate lr=50.0 or increase coordinate epsilon floor.

Security notes

All computations run strictly on local CPU memory without network transmission.

Extension exercises

  1. Implement Barnes-Hut quad-tree spatial acceleration.
  2. Benchmark t-SNE against UMAP on the MNIST handwriting dataset.
  • Lesson title: t-SNE and UMAP
  • Day number: 186 of 365
  • Lesson article: https://ai-roadmap-365.github.io/day-186-t-sne-and-umap
  • Lab files: everything you need is in this directory — follow “How to run” below.
  • Browse the course locally: from the repository root, this lab also appears in the course website at /labs/day-186-t-sne-and-umap when the site is running.

Expected output

FIELDS.md

# Expected Output Fields: Day 186

- `Input Shape`: Dimensions of input dataset (N, D).
- `Embedding Shape`: Dimensions of 2D embedding matrix (N, 2).
- `P Matrix Sum`: Total sum of joint probability matrix (should equal 1.0).

examples-run.txt

t-SNE Demo: Transformed (80, 4) to Embedding (80, 2)

measured-values.txt

Input Shape: (80, 4)
Embedding Shape: (80, 2)
P Matrix Sum: 1.0000

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 2 items

tests/test_t_sne_and_umap_lib.py::test_tsne_affinities_properties PASSED   [ 50%]
tests/test_t_sne_and_umap_lib.py::test_tsne_embedding_output_shape PASSED  [100%]

============================== 2 passed in 0.08s ===============================

Source files

examples/t_sne_and_umap_lib.py (2359 bytes) 
import numpy as np

class TSNEFromScratch:
    def __init__(self, n_components=2, perplexity=30.0, n_iter=300, lr=100.0, random_state=42):
        self.n_components = n_components
        self.perplexity = perplexity
        self.n_iter = n_iter
        self.lr = lr
        self.random_state = random_state
        self.embedding_ = None

    def _compute_affinities(self, X):
        n_samples = len(X)
        dists = np.linalg.norm(X[:, np.newaxis, :] - X[np.newaxis, :, :], axis=2)**2
        P = np.zeros((n_samples, n_samples))

        sigmas = np.median(dists, axis=1) / np.log(self.perplexity)
        for i in range(n_samples):
            num = np.exp(-dists[i] / (2.0 * sigmas[i] + 1e-12))
            num[i] = 0.0
            P[i] = num / (np.sum(num) + 1e-12)

        P = (P + P.T) / (2.0 * n_samples)
        P = np.maximum(P, 1e-12)
        return P

    def fit_transform(self, X):
        rng = np.random.default_rng(self.random_state)
        n_samples = len(X)
        P = self._compute_affinities(X)

        P_exagg = P * 4.0
        Y = rng.normal(0, 1e-4, (n_samples, self.n_components))
        velocity = np.zeros_like(Y)
        momentum = 0.5

        for step in range(self.n_iter):
            if step == 50:
                momentum = 0.8
                P_exagg = P

            dist_Y = np.linalg.norm(Y[:, np.newaxis, :] - Y[np.newaxis, :, :], axis=2)**2
            inv_dist = 1.0 / (1.0 + dist_Y)
            np.fill_diagonal(inv_dist, 0.0)
            Q = inv_dist / (np.sum(inv_dist) + 1e-12)
            Q = np.maximum(Q, 1e-12)

            PQ_diff = (P_exagg - Q) * inv_dist
            grad = np.zeros_like(Y)
            for i in range(n_samples):
                grad[i] = 4.0 * np.sum((Y[i] - Y) * PQ_diff[i, :, np.newaxis], axis=0)

            velocity = momentum * velocity - self.lr * grad
            Y += velocity

        self.embedding_ = Y
        return Y

def run_tsne_demo():
    np.random.seed(42)
    c1 = np.random.normal(loc=[-5.0, -5.0], scale=0.5, size=(40, 4))
    c2 = np.random.normal(loc=[5.0, 5.0], scale=0.5, size=(40, 4))
    X = np.vstack([c1, c2])

    tsne = TSNEFromScratch(n_components=2, perplexity=20.0, n_iter=100, lr=50.0).fit_transform(X)
    print(f"t-SNE Demo: Transformed {X.shape} to Embedding {tsne.shape}")
    return tsne

if __name__ == "__main__":
    run_tsne_demo()
examples/test_t_sne_and_umap_lib.py (679 bytes) 
import pytest
import numpy as np
from examples.t_sne_and_umap_lib import TSNEFromScratch

def test_tsne_affinities_properties():
    np.random.seed(42)
    X = np.random.normal(0, 1, (30, 3))
    tsne = TSNEFromScratch(perplexity=10.0)
    P = tsne._compute_affinities(X)

    assert P.shape == (30, 30)
    assert np.allclose(P, P.T)
    assert np.all(P >= 0)
    assert np.isclose(np.sum(P), 1.0, atol=1e-4)

def test_tsne_embedding_output_shape():
    np.random.seed(42)
    X = np.random.normal(0, 1, (40, 5))
    tsne = TSNEFromScratch(n_components=2, perplexity=10.0, n_iter=50)
    Y = tsne.fit_transform(X)

    assert Y.shape == (40, 2)
    assert not np.isnan(Y).any()
metadata.yml (427 bytes) 
lesson_id: D186
day: 186
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/t_sne_and_umap_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (62 bytes) 
numpy>=1.26.0
scikit-learn>=1.4.0
pytest>=8.0.0
scipy>=1.12.0
starter/t_sne_and_umap_lib.py (586 bytes) 
import numpy as np

class TSNEFromScratch:
    def __init__(self, n_components=2, perplexity=30.0, n_iter=300, lr=100.0, random_state=42):
        self.n_components = n_components
        self.perplexity = perplexity
        self.n_iter = n_iter
        self.lr = lr
        self.random_state = random_state
        self.embedding_ = None

    def _compute_affinities(self, X):
        # TODO: Compute Gaussian high-dimensional affinities
        pass

    def fit_transform(self, X):
        # TODO: Optimize low-dimensional coordinates Y using Student-t gradient descent
        pass
starter/test_t_sne_and_umap_lib.py (679 bytes) 
import pytest
import numpy as np
from examples.t_sne_and_umap_lib import TSNEFromScratch

def test_tsne_affinities_properties():
    np.random.seed(42)
    X = np.random.normal(0, 1, (30, 3))
    tsne = TSNEFromScratch(perplexity=10.0)
    P = tsne._compute_affinities(X)

    assert P.shape == (30, 30)
    assert np.allclose(P, P.T)
    assert np.all(P >= 0)
    assert np.isclose(np.sum(P), 1.0, atol=1e-4)

def test_tsne_embedding_output_shape():
    np.random.seed(42)
    X = np.random.normal(0, 1, (40, 5))
    tsne = TSNEFromScratch(n_components=2, perplexity=10.0, n_iter=50)
    Y = tsne.fit_transform(X)

    assert Y.shape == (40, 2)
    assert not np.isnan(Y).any()
tests/run_tests.sh (227 bytes) 
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 186 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_t_sne_and_umap_lib.py (679 bytes) 
import pytest
import numpy as np
from examples.t_sne_and_umap_lib import TSNEFromScratch

def test_tsne_affinities_properties():
    np.random.seed(42)
    X = np.random.normal(0, 1, (30, 3))
    tsne = TSNEFromScratch(perplexity=10.0)
    P = tsne._compute_affinities(X)

    assert P.shape == (30, 30)
    assert np.allclose(P, P.T)
    assert np.all(P >= 0)
    assert np.isclose(np.sum(P), 1.0, atol=1e-4)

def test_tsne_embedding_output_shape():
    np.random.seed(42)
    X = np.random.normal(0, 1, (40, 5))
    tsne = TSNEFromScratch(n_components=2, perplexity=10.0, n_iter=50)
    Y = tsne.fit_transform(X)

    assert Y.shape == (40, 2)
    assert not np.isnan(Y).any()

Troubleshooting

Troubleshooting: Day 186 - t-SNE and UMAP

Common Issues

  1. Coordinate Drift / Overlap:
    • Handled by early exaggeration scaling during the initial 50 iterations.

Security notes

Security & Privacy: Day 186 - t-SNE and UMAP

Security Guidance

  • All calculations execute locally without third-party network egress.