Deep Learning › Convolutional Networks and Vision › Day 216

Hands-on lab — Day 216: Beyond Classification: Detection and Segmentation

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/beyond_classification_detection_and_segmentation_lib.py

Test

./tests/run_tests.sh

File tree

conftest.py
examples/beyond_classification_detection_and_segmentation_lib.py
examples/test_beyond_classification_detection_and_segmentation_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/beyond_classification_detection_and_segmentation_lib.py
starter/test_beyond_classification_detection_and_segmentation_lib.py
tests/run_tests.sh
tests/test_beyond_classification_detection_and_segmentation_lib.py
troubleshooting.md

Lab README

Lab: Day 216 -- Beyond Classification: Detection and Segmentation

Lesson

Day number: 216 of 365. Course: Course05-SS01 (Deep Learning - Neural Networks). Topic: Object Detection and Semantic Segmentation Algorithms in PyTorch.

Purpose

Build and test a modular dense spatial computer vision library in PyTorch. Implement Intersection over Union (IoU) bounding box calculations, Non-Maximum Suppression (NMS) candidate filtering, and soft Dice Loss for imbalanced segmentation masks.

Learning objectives

  • Implement bounding box Intersection over Union (IoU) from coordinate geometry.
  • Implement greedy Non-Maximum Suppression (NMS) to eliminate duplicate detection proposals.
  • Implement soft Dice Loss for binary semantic segmentation masks.
  • Validate spatial prediction accuracy on multi-object test batches.

Prerequisites

  • Day 215 (Training a Vision Model End to End).
  • Python 3.11+ with PyTorch.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 1 GB RAM.
  • 100 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

PyTorch is free and open-source under the modified BSD license.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/beyond_classification_detection_and_segmentation_lib.py: Student scaffold file.
  • examples/beyond_classification_detection_and_segmentation_lib.py: Complete reference implementation.
  • tests/test_beyond_classification_detection_and_segmentation_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/beyond_classification_detection_and_segmentation_lib.py

What the commands do

  • Computes IoU between overlapping bounding boxes.
  • Filters redundant candidate proposals using NMS.
  • Runs validation unit tests.

Expected output

Dense Vision Demo: IoU = 0.3333, NMS Kept Indices = [0, 2]

Validation steps

  1. Verify calculate_iou yields 1.0 for identical boxes and 0.0 for disjoint boxes.
  2. Confirm apply_nms removes overlapping duplicate boxes above the threshold.
  3. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Negative IoU Values: Ensure coordinate differences are clamped with max(0.0, ...).

Security notes

All spatial algorithms execute locally in CPU memory without external network calls.

Extension exercises

  1. Implement Generalized IoU (GIoU) and Complete IoU (CIoU) bounding box loss functions.
  2. Implement Mean Average Precision (mAP@0.5) evaluator over detection batches.

Expected output

FIELDS.md

# Expected Output Fields: Day 216

- `IoU`: Intersection over Union scalar value.
- `NMS Kept Indices`: List of box indices retained after Non-Maximum Suppression.

examples-run.txt

Dense Vision Demo: IoU = 0.3333, NMS Kept Indices = [0, 2]

measured-values.txt

IoU: 0.3333
NMS Kept Indices: [0, 2]

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 4 items

tests/test_beyond_classification_detection_and_segmentation_lib.py::test_calculate_iou_exact PASSED [ 25%]
tests/test_beyond_classification_detection_and_segmentation_lib.py::test_calculate_iou_disjoint PASSED [ 50%]
tests/test_beyond_classification_detection_and_segmentation_lib.py::test_apply_nms_suppression PASSED [ 75%]
tests/test_beyond_classification_detection_and_segmentation_lib.py::test_dice_loss_perfect PASSED   [100%]

============================== 4 passed in 0.32s ===============================

Source files

conftest.py (585 bytes)
"""Make this lab importable when the suite runs.

pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""

import os
import sys

LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
    if os.path.isdir(candidate) and candidate not in sys.path:
        sys.path.insert(0, candidate)
examples/beyond_classification_detection_and_segmentation_lib.py (2576 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any, List

def calculate_iou(box_a: torch.Tensor, box_b: torch.Tensor) -> float:
    # Box format: [x1, y1, x2, y2]
    x1 = max(float(box_a[0]), float(box_b[0]))
    y1 = max(float(box_a[1]), float(box_b[1]))
    x2 = min(float(box_a[2]), float(box_b[2]))
    y2 = min(float(box_a[3]), float(box_b[3]))

    inter_w = max(0.0, x2 - x1)
    inter_h = max(0.0, y2 - y1)
    inter_area = inter_w * inter_h

    area_a = max(0.0, float(box_a[2] - box_a[0])) * max(0.0, float(box_a[3] - box_a[1]))
    area_b = max(0.0, float(box_b[2] - box_b[0])) * max(0.0, float(box_b[3] - box_b[1]))

    union_area = area_a + area_b - inter_area
    if union_area == 0.0:
        return 0.0
    return inter_area / union_area

def apply_nms(boxes: torch.Tensor, scores: torch.Tensor, iou_threshold: float = 0.5) -> torch.Tensor:
    # boxes: (N, 4) in [x1, y1, x2, y2], scores: (N,)
    if boxes.size(0) == 0:
        return torch.empty(0, dtype=torch.long)

    order = torch.argsort(scores, descending=True)
    keep = []

    while order.numel() > 0:
        i = order[0].item()
        keep.append(i)

        if order.numel() == 1:
            break

        # Compute IoU of the picked box with all remaining boxes
        remaining_indices = order[1:]
        ious = torch.tensor([
            calculate_iou(boxes[i], boxes[j]) for j in remaining_indices
        ])

        # Keep boxes with IoU <= threshold
        mask = ious <= iou_threshold
        order = remaining_indices[mask]

    return torch.tensor(keep, dtype=torch.long)

def calculate_dice_loss(pred_probs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-6) -> float:
    # pred_probs: (N, H, W) in [0, 1], targets: (N, H, W) binary
    intersection = (pred_probs * targets).sum()
    cardinality = pred_probs.sum() + targets.sum()
    dice_score = (2.0 * intersection + eps) / (cardinality + eps)
    return float(1.0 - dice_score)

def run_dense_vision_demo():
    box1 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    box2 = torch.tensor([5.0, 0.0, 15.0, 10.0]) # 50% overlap

    iou = calculate_iou(box1, box2)

    boxes = torch.tensor([
        [0.0, 0.0, 10.0, 10.0],
        [1.0, 1.0, 10.0, 10.0], # High overlap duplicate
        [20.0, 20.0, 30.0, 30.0] # Separate object
    ])
    scores = torch.tensor([0.9, 0.7, 0.85])
    keep = apply_nms(boxes, scores, iou_threshold=0.5)

    print(f"Dense Vision Demo: IoU = {iou:.4f}, NMS Kept Indices = {keep.tolist()}")
    return iou, keep

if __name__ == "__main__":
    run_dense_vision_demo()
examples/test_beyond_classification_detection_and_segmentation_lib.py (978 bytes)
import pytest
import torch
from examples.beyond_classification_detection_and_segmentation_lib import calculate_iou, apply_nms, calculate_dice_loss

def test_calculate_iou_exact():
    box1 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    box2 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    assert calculate_iou(box1, box2) == 1.0

def test_calculate_iou_disjoint():
    box1 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    box2 = torch.tensor([20.0, 20.0, 30.0, 30.0])
    assert calculate_iou(box1, box2) == 0.0

def test_apply_nms_suppression():
    boxes = torch.tensor([
        [0.0, 0.0, 10.0, 10.0],
        [1.0, 1.0, 10.0, 10.0], # Overlapping duplicate
        [50.0, 50.0, 60.0, 60.0] # Separate object
    ])
    scores = torch.tensor([0.95, 0.80, 0.88])
    keep = apply_nms(boxes, scores, iou_threshold=0.5)

    assert keep.tolist() == [0, 2]

def test_dice_loss_perfect():
    mask = torch.ones(2, 16, 16)
    loss = calculate_dice_loss(mask, mask)
    assert abs(loss) < 1e-4
metadata.yml (461 bytes)
lesson_id: D216
day: 216
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/beyond_classification_detection_and_segmentation_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (27 bytes)
torch>=2.2.0
pytest>=8.0.0
starter/beyond_classification_detection_and_segmentation_lib.py (549 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any, List

def calculate_iou(box_a: torch.Tensor, box_b: torch.Tensor) -> float:
    # TODO: Implement 2D Bounding Box Intersection over Union
    pass

def apply_nms(boxes: torch.Tensor, scores: torch.Tensor, iou_threshold: float = 0.5) -> torch.Tensor:
    # TODO: Implement Non-Maximum Suppression
    pass

def calculate_dice_loss(pred_probs: torch.Tensor, targets: torch.Tensor, eps: float = 1e-6) -> float:
    # TODO: Implement Soft Dice Loss for segmentation masks
    pass
starter/test_beyond_classification_detection_and_segmentation_lib.py (978 bytes)
import pytest
import torch
from examples.beyond_classification_detection_and_segmentation_lib import calculate_iou, apply_nms, calculate_dice_loss

def test_calculate_iou_exact():
    box1 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    box2 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    assert calculate_iou(box1, box2) == 1.0

def test_calculate_iou_disjoint():
    box1 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    box2 = torch.tensor([20.0, 20.0, 30.0, 30.0])
    assert calculate_iou(box1, box2) == 0.0

def test_apply_nms_suppression():
    boxes = torch.tensor([
        [0.0, 0.0, 10.0, 10.0],
        [1.0, 1.0, 10.0, 10.0], # Overlapping duplicate
        [50.0, 50.0, 60.0, 60.0] # Separate object
    ])
    scores = torch.tensor([0.95, 0.80, 0.88])
    keep = apply_nms(boxes, scores, iou_threshold=0.5)

    assert keep.tolist() == [0, 2]

def test_dice_loss_perfect():
    mask = torch.ones(2, 16, 16)
    loss = calculate_dice_loss(mask, mask)
    assert abs(loss) < 1e-4
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 216 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_beyond_classification_detection_and_segmentation_lib.py (978 bytes)
import pytest
import torch
from examples.beyond_classification_detection_and_segmentation_lib import calculate_iou, apply_nms, calculate_dice_loss

def test_calculate_iou_exact():
    box1 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    box2 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    assert calculate_iou(box1, box2) == 1.0

def test_calculate_iou_disjoint():
    box1 = torch.tensor([0.0, 0.0, 10.0, 10.0])
    box2 = torch.tensor([20.0, 20.0, 30.0, 30.0])
    assert calculate_iou(box1, box2) == 0.0

def test_apply_nms_suppression():
    boxes = torch.tensor([
        [0.0, 0.0, 10.0, 10.0],
        [1.0, 1.0, 10.0, 10.0], # Overlapping duplicate
        [50.0, 50.0, 60.0, 60.0] # Separate object
    ])
    scores = torch.tensor([0.95, 0.80, 0.88])
    keep = apply_nms(boxes, scores, iou_threshold=0.5)

    assert keep.tolist() == [0, 2]

def test_dice_loss_perfect():
    mask = torch.ones(2, 16, 16)
    loss = calculate_dice_loss(mask, mask)
    assert abs(loss) < 1e-4

Troubleshooting

Troubleshooting: Day 216 - Beyond Classification

Common Issues

  1. Division by zero in IoU:
    • Cause: Both boxes have zero area.
    • Fix: Return 0.0 if union_area == 0.0.

Security notes

Security & Privacy: Day 216 - Beyond Classification

Security Guidance

  • All bounding box and mask calculations run locally in CPU memory.