Deep Learning › Convolutional Networks and Vision › Day 215

Hands-on lab — Day 215: Training a Vision Model End to End

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/training_a_vision_model_end_to_lib.py

Test

./tests/run_tests.sh

File tree

conftest.py
examples/test_training_a_vision_model_end_to_lib.py
examples/training_a_vision_model_end_to_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/test_training_a_vision_model_end_to_lib.py
starter/training_a_vision_model_end_to_lib.py
tests/run_tests.sh
tests/test_training_a_vision_model_end_to_lib.py
troubleshooting.md

Lab README

Lab: Day 215 -- Training a Vision Model End to End

Lesson

Day number: 215 of 365. Course: Course05-SS01 (Deep Learning - Neural Networks). Topic: End-to-End Vision Training and Evaluation in PyTorch.

Purpose

Build and test an industrial-grade Computer Vision training and evaluation library in PyTorch. Implement Top-1 and Top-K classification accuracy evaluators, construct normalized confusion matrix calculators, and verify metric correctness against multi-class logit distributions.

Learning objectives

  • Calculate Top-1 and Top-K classification accuracy from raw model logits.
  • Compute normalized multi-class confusion matrices.
  • Analyze per-class recall and pairwise misclassification patterns.
  • Implement robust evaluation routines without gradient memory leaks.

Prerequisites

  • Day 214 (Data Augmentation).
  • Python 3.11+ with PyTorch.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 1 GB RAM.
  • 100 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

PyTorch is free and open-source under the modified BSD license.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/training_a_vision_model_end_to_lib.py: Student scaffold file.
  • examples/training_a_vision_model_end_to_lib.py: Complete reference implementation.
  • tests/test_training_a_vision_model_end_to_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/training_a_vision_model_end_to_lib.py

What the commands do

  • Evaluates Top-1 and Top-3 accuracy on synthetic logit distributions.
  • Computes normalized confusion matrix diagonal.
  • Runs validation test suite.

Expected output

Training Demo: Top-1 = 1.00, Top-3 = 1.00, CM Diag = [1.0, 1.0, 1.0, 1.0, 1.0]

Validation steps

  1. Confirm calculate_topk_accuracy accurately counts Top-1 and Top-K inclusion.
  2. Confirm compute_confusion_matrix normalizes rows to sum to 1.0.
  3. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Accuracy greater than 1.0: Verify division by float targets.size(0).

Security notes

All training metrics execute in local system memory without external telemetry.

Extension exercises

  1. Implement Exponential Moving Average (EMA) weight tracking.
  2. Implement an automated Early Stopping class restoring best checkpoint weights.

Expected output

FIELDS.md

# Expected Output Fields: Day 215

- `Top-1`: Top-1 classification accuracy.
- `Top-3`: Top-3 classification accuracy.
- `CM Diag`: Diagonal elements of the normalized confusion matrix.

examples-run.txt

Training Demo: Top-1 = 1.00, Top-3 = 1.00, CM Diag = [1.0, 1.0, 1.0, 1.0, 1.0]

measured-values.txt

Top-1: 1.00
Top-3: 1.00
CM Diag: [1.0, 1.0, 1.0, 1.0, 1.0]

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 3 items

tests/test_training_a_vision_model_end_to_lib.py::test_topk_accuracy_top1 PASSED [ 33%]
tests/test_training_a_vision_model_end_to_lib.py::test_topk_accuracy_top2 PASSED [ 66%]
tests/test_training_a_vision_model_end_to_lib.py::test_confusion_matrix_diagonal_perfect PASSED [100%]

============================== 3 passed in 0.34s ===============================

Source files

conftest.py (585 bytes)
"""Make this lab importable when the suite runs.

pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""

import os
import sys

LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
    if os.path.isdir(candidate) and candidate not in sys.path:
        sys.path.insert(0, candidate)
examples/test_training_a_vision_model_end_to_lib.py (1109 bytes)
import pytest
import torch
from examples.training_a_vision_model_end_to_lib import calculate_topk_accuracy, compute_confusion_matrix

def test_topk_accuracy_top1():
    logits = torch.tensor([
        [5.0, 1.0, 0.0],
        [0.0, 5.0, 1.0],
        [1.0, 0.0, 5.0],
        [5.0, 1.0, 0.0]
    ])
    targets = torch.tensor([0, 1, 2, 1]) # Last sample is incorrect (pred 0, true 1)

    top1 = calculate_topk_accuracy(logits, targets, k=1)
    assert top1 == 0.75

def test_topk_accuracy_top2():
    logits = torch.tensor([
        [5.0, 1.0, 0.0],
        [0.0, 5.0, 1.0],
        [1.0, 0.0, 5.0],
        [5.0, 4.0, 0.0]  # Sample 4: rank1=0, rank2=1. True is 1 -> included in Top-2!
    ])
    targets = torch.tensor([0, 1, 2, 1])

    top2 = calculate_topk_accuracy(logits, targets, k=2)
    assert top2 == 1.0

def test_confusion_matrix_diagonal_perfect():
    preds = torch.tensor([0, 1, 2, 0, 1, 2])
    targets = torch.tensor([0, 1, 2, 0, 1, 2])

    cm = compute_confusion_matrix(preds, targets, num_classes=3)
    # Diagonal should all be 1.0
    for i in range(3):
        assert cm[i, i] == 1.0
examples/training_a_vision_model_end_to_lib.py (1608 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any, List

def calculate_topk_accuracy(logits: torch.Tensor, targets: torch.Tensor, k: int = 5) -> float:
    with torch.no_grad():
        k = min(k, logits.size(1))
        _, pred = logits.topk(k, dim=1, largest=True, sorted=True)
        correct = pred.eq(targets.view(-1, 1).expand_as(pred))
        topk_correct = float(correct.sum().item())
        return topk_correct / float(targets.size(0))

def compute_confusion_matrix(preds: torch.Tensor, targets: torch.Tensor, num_classes: int) -> torch.Tensor:
    with torch.no_grad():
        cm = torch.zeros(num_classes, num_classes, dtype=torch.float32)
        for p, t in zip(preds, targets):
            cm[t.item(), p.item()] += 1.0

        # Row normalize
        row_sums = cm.sum(dim=1, keepdim=True)
        row_sums[row_sums == 0] = 1.0
        cm_norm = cm / row_sums
        return cm_norm

def run_vision_training_demo():
    torch.manual_seed(42)
    logits = torch.tensor([
        [2.5, 0.1, 0.3, 0.8, 0.2],
        [0.1, 3.2, 0.4, 0.5, 0.2],
        [0.2, 0.3, 1.1, 2.8, 0.5],
        [0.1, 0.2, 0.3, 0.4, 4.0]
    ])
    targets = torch.tensor([0, 1, 3, 4])

    top1 = calculate_topk_accuracy(logits, targets, k=1)
    top3 = calculate_topk_accuracy(logits, targets, k=3)
    preds = torch.argmax(logits, dim=1)
    cm = compute_confusion_matrix(preds, targets, num_classes=5)

    print(f"Training Demo: Top-1 = {top1:.2f}, Top-3 = {top3:.2f}, CM Diag = {torch.diag(cm).tolist()}")
    return top1, top3, cm

if __name__ == "__main__":
    run_vision_training_demo()
metadata.yml (443 bytes)
lesson_id: D215
day: 215
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/training_a_vision_model_end_to_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (27 bytes)
torch>=2.2.0
pytest>=8.0.0
starter/test_training_a_vision_model_end_to_lib.py (1109 bytes)
import pytest
import torch
from examples.training_a_vision_model_end_to_lib import calculate_topk_accuracy, compute_confusion_matrix

def test_topk_accuracy_top1():
    logits = torch.tensor([
        [5.0, 1.0, 0.0],
        [0.0, 5.0, 1.0],
        [1.0, 0.0, 5.0],
        [5.0, 1.0, 0.0]
    ])
    targets = torch.tensor([0, 1, 2, 1]) # Last sample is incorrect (pred 0, true 1)

    top1 = calculate_topk_accuracy(logits, targets, k=1)
    assert top1 == 0.75

def test_topk_accuracy_top2():
    logits = torch.tensor([
        [5.0, 1.0, 0.0],
        [0.0, 5.0, 1.0],
        [1.0, 0.0, 5.0],
        [5.0, 4.0, 0.0]  # Sample 4: rank1=0, rank2=1. True is 1 -> included in Top-2!
    ])
    targets = torch.tensor([0, 1, 2, 1])

    top2 = calculate_topk_accuracy(logits, targets, k=2)
    assert top2 == 1.0

def test_confusion_matrix_diagonal_perfect():
    preds = torch.tensor([0, 1, 2, 0, 1, 2])
    targets = torch.tensor([0, 1, 2, 0, 1, 2])

    cm = compute_confusion_matrix(preds, targets, num_classes=3)
    # Diagonal should all be 1.0
    for i in range(3):
        assert cm[i, i] == 1.0
starter/training_a_vision_model_end_to_lib.py (412 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any, List

def calculate_topk_accuracy(logits: torch.Tensor, targets: torch.Tensor, k: int = 5) -> float:
    # TODO: Calculate Top-K classification accuracy
    pass

def compute_confusion_matrix(preds: torch.Tensor, targets: torch.Tensor, num_classes: int) -> torch.Tensor:
    # TODO: Compute normalized multi-class confusion matrix
    pass
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 215 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_training_a_vision_model_end_to_lib.py (1109 bytes)
import pytest
import torch
from examples.training_a_vision_model_end_to_lib import calculate_topk_accuracy, compute_confusion_matrix

def test_topk_accuracy_top1():
    logits = torch.tensor([
        [5.0, 1.0, 0.0],
        [0.0, 5.0, 1.0],
        [1.0, 0.0, 5.0],
        [5.0, 1.0, 0.0]
    ])
    targets = torch.tensor([0, 1, 2, 1]) # Last sample is incorrect (pred 0, true 1)

    top1 = calculate_topk_accuracy(logits, targets, k=1)
    assert top1 == 0.75

def test_topk_accuracy_top2():
    logits = torch.tensor([
        [5.0, 1.0, 0.0],
        [0.0, 5.0, 1.0],
        [1.0, 0.0, 5.0],
        [5.0, 4.0, 0.0]  # Sample 4: rank1=0, rank2=1. True is 1 -> included in Top-2!
    ])
    targets = torch.tensor([0, 1, 2, 1])

    top2 = calculate_topk_accuracy(logits, targets, k=2)
    assert top2 == 1.0

def test_confusion_matrix_diagonal_perfect():
    preds = torch.tensor([0, 1, 2, 0, 1, 2])
    targets = torch.tensor([0, 1, 2, 0, 1, 2])

    cm = compute_confusion_matrix(preds, targets, num_classes=3)
    # Diagonal should all be 1.0
    for i in range(3):
        assert cm[i, i] == 1.0

Troubleshooting

Troubleshooting: Day 215 - Training a Vision Model End to End

Common Issues

  1. Memory leak during validation:
    • Cause: Computing metrics without torch.no_grad().
    • Fix: Always wrap evaluation loops in with torch.no_grad():.

Security notes

Security & Privacy: Day 215 - Training a Vision Model End to End

Security Guidance

  • All vision model training runs locally without transmitting data over the network.