Machine Learning › Unsupervised Learning › Day 185
Hands-on lab — Day 185: Principal Component Analysis
- ← Back to the Day 185 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/machine-learning/day-185-principal-component-analysis/
Commands
Setup
pip install -r requirements/requirements.txtRun
python3 examples/principal_component_analysis_lib.pyTest
./tests/run_tests.shFile tree
examples/principal_component_analysis_lib.py examples/test_principal_component_analysis_lib.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/starter-run.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/principal_component_analysis_lib.py starter/test_principal_component_analysis_lib.py tests/run_tests.sh tests/test_principal_component_analysis_lib.py troubleshooting.md
Lab README
Lab: Day 185 -- Principal Component Analysis
Lesson
Day number: 185 of 365. Course: Course04-SS03 (Beyond Supervised Learning). Topic: Principal Component Analysis and Linear Dimensionality Reduction.
Purpose
Build a complete, pure NumPy implementation of Principal Component Analysis from scratch using Singular Value Decomposition (SVD). You will implement data mean centering, singular vector extraction, explained variance ratio calculation, projection transformations, and inverse reconstruction.
Learning objectives
- Implement zero-mean data centering and economy SVD.
- Derive covariance eigenvalues from SVD singular values.
- Calculate Explained Variance Ratios and cumulative variance profiles.
- Transform high-dimensional data onto orthogonal principal axes and invert projections.
Prerequisites
- Linear algebra: Matrix multiplication, orthogonal matrices, SVD factorizations.
- Python 3.11+ with NumPy.
Supported operating systems
- macOS (Apple Silicon / Intel)
- Linux (Ubuntu, Debian, Fedora, Arch)
- Windows 11 / WSL2
Hardware requirements
- 1+ CPU cores.
- 512 MB RAM.
- 50 MB disk space.
Required software
- Python 3.11 or newer.
- pip package manager.
- virtualenv or venv module.
Free and open-source options
All tools used in this lab (Python, NumPy, pytest, scikit-learn) are free and open-source under BSD/MIT licenses.
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/principal_component_analysis_lib.py: Student scaffold file.examples/principal_component_analysis_lib.py: Complete reference implementation.tests/test_principal_component_analysis_lib.py: Pytest automated validation suite.expected-output/: Verified output logs and baseline values.
How to run
Execute the reference demonstration script:
python3 examples/principal_component_analysis_lib.py
What the commands do
- Generates a synthetic 5D dataset with correlated feature dimensions.
- Fits
PCAFromScratchto compress data from 5D to 2D. - Reconstructs original coordinates and computes Mean Squared Error.
Expected output
PCA Demo: Input Shape (200, 5) -> Reduced (200, 2), Reconstruction MSE = 0.142
Validation steps
- Verify that
components_vectors are orthonormal (V^T V = I). - Check that
explained_variance_ratio_sums to <= 1.0. - Verify that retaining all components yields near-zero reconstruction MSE (< 1e-5).
Tests
Run the test runner script:
./tests/run_tests.sh
Cleanup
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +
Troubleshooting
- Reconstruction Bias: Ensure
self.mean_is added back duringinverse_transform().
Security notes
All computations execute strictly on local CPU memory.
Extension exercises
- Implement Whitening and verify that transformed features have identity covariance.
- Build an automated Scree Plot generator in matplotlib.
Navigation
- Lesson title: Principal Component Analysis
- Day number: 185 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-185-principal-component-analysis
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-185-principal-component-analysiswhen the site is running.
Expected output
FIELDS.md
# Expected Output Fields: Day 185
- `Input Shape`: Tuple dimensions of original matrix (N, D).
- `Reduced Shape`: Tuple dimensions of transformed matrix (N, k).
- `Reconstruction MSE`: Mean squared error between X and inverse transformed X.
examples-run.txt
PCA Demo: Input Shape (200, 5) -> Reduced (200, 2), Reconstruction MSE = 0.1420
measured-values.txt
Input Shape: (200, 5)
Reduced Shape: (200, 2)
Reconstruction MSE: 0.1420
starter-run.txt
Starter scaffold executed. Ready for student implementation.
test-run.txt
============================= test session starts ==============================
collected 2 items
tests/test_principal_component_analysis_lib.py::test_pca_shapes_and_explained_variance PASSED [ 50%]
tests/test_principal_component_analysis_lib.py::test_pca_reconstruction PASSED [100%]
============================== 2 passed in 0.08s ===============================
Source files
examples/principal_component_analysis_lib.py (1845 bytes)
import numpy as np
class PCAFromScratch:
def __init__(self, n_components=2, whiten=False):
self.n_components = n_components
self.whiten = whiten
self.components_ = None
self.explained_variance_ = None
self.explained_variance_ratio_ = None
self.mean_ = None
def fit(self, X):
n_samples, n_features = X.shape
self.mean_ = np.mean(X, axis=0)
X_centered = X - self.mean_
U, S, Vt = np.linalg.svd(X_centered, full_matrices=False)
explained_variance = (S ** 2) / (n_samples - 1)
total_variance = np.sum(explained_variance)
explained_variance_ratio = explained_variance / total_variance
self.components_ = Vt[:self.n_components]
self.explained_variance_ = explained_variance[:self.n_components]
self.explained_variance_ratio_ = explained_variance_ratio[:self.n_components]
return self
def transform(self, X):
X_centered = X - self.mean_
Z = np.dot(X_centered, self.components_.T)
if self.whiten:
scale = np.sqrt(self.explained_variance_) + 1e-12
Z = Z / scale
return Z
def inverse_transform(self, Z):
if self.whiten:
scale = np.sqrt(self.explained_variance_) + 1e-12
Z = Z * scale
return np.dot(Z, self.components_) + self.mean_
def run_pca_demo():
np.random.seed(42)
X = np.random.normal(0, 1, (200, 5))
X[:, 1] = X[:, 0] * 2.0 + np.random.normal(0, 0.2, 200)
pca = PCAFromScratch(n_components=2).fit(X)
Z = pca.transform(X)
X_rec = pca.inverse_transform(Z)
mse = float(np.mean((X - X_rec) ** 2))
print(f"PCA Demo: Input Shape {X.shape} -> Reduced {Z.shape}, Reconstruction MSE = {mse:.4f}")
return pca, Z, mse
if __name__ == "__main__":
run_pca_demo()
examples/test_principal_component_analysis_lib.py (742 bytes)
import pytest
import numpy as np
from examples.principal_component_analysis_lib import PCAFromScratch
def test_pca_shapes_and_explained_variance():
np.random.seed(42)
X = np.random.normal(0, 1, (100, 4))
pca = PCAFromScratch(n_components=2).fit(X)
assert pca.components_.shape == (2, 4)
assert len(pca.explained_variance_ratio_) == 2
assert np.sum(pca.explained_variance_ratio_) <= 1.0
assert pca.explained_variance_ratio_[0] >= pca.explained_variance_ratio_[1]
def test_pca_reconstruction():
np.random.seed(42)
X = np.random.normal(0, 1, (50, 3))
pca = PCAFromScratch(n_components=3).fit(X)
Z = pca.transform(X)
X_rec = pca.inverse_transform(Z)
assert np.allclose(X, X_rec, atol=1e-5)
metadata.yml (441 bytes)
lesson_id: D185
day: 185
kind: lab
languages:
- python
setup_commands:
- 'pip install -r requirements/requirements.txt'
run_commands:
- 'python3 examples/principal_component_analysis_lib.py'
test_commands:
- './tests/run_tests.sh'
cleanup_commands:
- 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (62 bytes)
numpy>=1.26.0
scikit-learn>=1.4.0
pytest>=8.0.0
scipy>=1.12.0
starter/principal_component_analysis_lib.py (638 bytes)
import numpy as np
class PCAFromScratch:
def __init__(self, n_components=2, whiten=False):
self.n_components = n_components
self.whiten = whiten
self.components_ = None
self.explained_variance_ = None
self.explained_variance_ratio_ = None
self.mean_ = None
def fit(self, X):
# TODO: Implement zero-centering, SVD decomposition, and eigenvalue extraction
pass
def transform(self, X):
# TODO: Project centered data onto principal components
pass
def inverse_transform(self, Z):
# TODO: Reconstruct original coordinates
pass
starter/test_principal_component_analysis_lib.py (742 bytes)
import pytest
import numpy as np
from examples.principal_component_analysis_lib import PCAFromScratch
def test_pca_shapes_and_explained_variance():
np.random.seed(42)
X = np.random.normal(0, 1, (100, 4))
pca = PCAFromScratch(n_components=2).fit(X)
assert pca.components_.shape == (2, 4)
assert len(pca.explained_variance_ratio_) == 2
assert np.sum(pca.explained_variance_ratio_) <= 1.0
assert pca.explained_variance_ratio_[0] >= pca.explained_variance_ratio_[1]
def test_pca_reconstruction():
np.random.seed(42)
X = np.random.normal(0, 1, (50, 3))
pca = PCAFromScratch(n_components=3).fit(X)
Z = pca.transform(X)
X_rec = pca.inverse_transform(Z)
assert np.allclose(X, X_rec, atol=1e-5)
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 185 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_principal_component_analysis_lib.py (742 bytes)
import pytest
import numpy as np
from examples.principal_component_analysis_lib import PCAFromScratch
def test_pca_shapes_and_explained_variance():
np.random.seed(42)
X = np.random.normal(0, 1, (100, 4))
pca = PCAFromScratch(n_components=2).fit(X)
assert pca.components_.shape == (2, 4)
assert len(pca.explained_variance_ratio_) == 2
assert np.sum(pca.explained_variance_ratio_) <= 1.0
assert pca.explained_variance_ratio_[0] >= pca.explained_variance_ratio_[1]
def test_pca_reconstruction():
np.random.seed(42)
X = np.random.normal(0, 1, (50, 3))
pca = PCAFromScratch(n_components=3).fit(X)
Z = pca.transform(X)
X_rec = pca.inverse_transform(Z)
assert np.allclose(X, X_rec, atol=1e-5)
Troubleshooting
Troubleshooting: Day 185 - Principal Component Analysis
Common Issues
- Uncentered Data Distortions:
- Cause: Forgetting to subtract column means before running SVD.
- Fix: Apply
X_centered = X - self.mean_.
Security notes
Security & Privacy: Day 185 - Principal Component Analysis
Security Guidance
- All computations run locally without network transmission.