Deep Learning › Convolutional Networks and Vision › Day 211

Hands-on lab — Day 211: Convolutions

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/convolutions_lib.py

Test

./tests/run_tests.sh

File tree

conftest.py
examples/convolutions_lib.py
examples/test_convolutions_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/convolutions_lib.py
starter/test_convolutions_lib.py
tests/run_tests.sh
tests/test_convolutions_lib.py
troubleshooting.md

Lab README

Lab: Day 211 -- Convolutions

Lesson

Day number: 211 of 365. Course: Course05-SS01 (Deep Learning - Neural Networks). Topic: Convolutions in PyTorch and NumPy.

Purpose

Build and test a multi-channel 2D convolution engine from scratch in pure NumPy. Verify mathematical equivalence with PyTorch's native nn.Conv2d, calculate output spatial dimensions under various strides and paddings, and apply classical Sobel spatial filter kernels.

Learning objectives

  • Implement 2D cross-correlation sliding window operations with multiple input and output channels.
  • Implement spatial padding and strided convolution logic.
  • Verify bitwise numerical alignment against torch.nn.functional.conv2d.
  • Apply classical Sobel edge detection kernels to image tensors.

Prerequisites

  • Day 210 (A Disciplined Training Project).
  • Python 3.11+ with PyTorch and NumPy.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 1 GB RAM.
  • 100 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

PyTorch and NumPy are free and open-source under modified BSD licenses.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/convolutions_lib.py: Student scaffold file.
  • examples/convolutions_lib.py: Complete reference implementation.
  • tests/test_convolutions_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/convolutions_lib.py

What the commands do

  • Executes a multi-channel 2D convolution across synthetic image tensors.
  • Compares NumPy implementation outputs against PyTorch native F.conv2d.
  • Verifies edge detection responses.

Expected output

2D Conv Demo: Output Shape = (2, 4, 16, 16), Max Diff vs PyTorch = 0.000000e+00

Validation steps

  1. Verify that calculate_conv_output_dim computes exact spatial resolutions.
  2. Confirm that custom_conv2d_numpy matches F.conv2d within 1e-5 numerical tolerance.
  3. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Dimensions Mismatch: Verify (H - K + 2*P) // S + 1 uses integer division.

Security notes

All calculations run in local system memory on CPU hardware.

Extension exercises

  1. Implement Depthwise Separable Convolution in pure NumPy.
  2. Build a Dilated Convolution engine supporting arbitrary dilation factors.
  • Lesson title: Convolutions
  • Day number: 211 of 365
  • Lesson article: https://ai-roadmap-365.github.io/day-211-convolutions
  • Lab files: everything you need is in this directory — follow “How to run” below.
  • Browse the course locally: from the repository root, this lab also appears in the course website at /labs/day-211-convolutions when the site is running.

Expected output

FIELDS.md

# Expected Output Fields: Day 211

- `Output Shape`: 4D tuple of output tensor dimensions (N, C_out, H_out, W_out).
- `Max Diff vs PyTorch`: Maximum absolute difference against torch.nn.functional.conv2d.

examples-run.txt

2D Conv Demo: Output Shape = (2, 4, 16, 16), Max Diff vs PyTorch = 0.000000e+00

measured-values.txt

Output Shape: (2, 4, 16, 16)
Max Diff vs PyTorch: 0.000000e+00

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 3 items

tests/test_convolutions_lib.py::test_calculate_conv_output_dim PASSED    [ 33%]
tests/test_convolutions_lib.py::test_custom_conv2d_numpy_matches_pytorch PASSED [ 66%]
tests/test_convolutions_lib.py::test_sobel_edge_detection_filter PASSED   [100%]

============================== 3 passed in 0.26s ===============================

Source files

conftest.py (585 bytes)
"""Make this lab importable when the suite runs.

pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""

import os
import sys

LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
    if os.path.isdir(candidate) and candidate not in sys.path:
        sys.path.insert(0, candidate)
examples/convolutions_lib.py (2259 bytes)
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import Tuple, Dict, Any

def calculate_conv_output_dim(h_in: int, w_in: int, k: int,
                              stride: int = 1, padding: int = 0) -> Tuple[int, int]:
    h_out = ((h_in - k + 2 * padding) // stride) + 1
    w_out = ((w_in - k + 2 * padding) // stride) + 1
    return h_out, w_out

def custom_conv2d_numpy(x: np.ndarray, weight: np.ndarray, bias: np.ndarray = None,
                         stride: int = 1, padding: int = 0) -> np.ndarray:
    # x shape: (N, C_in, H, W)
    # weight shape: (C_out, C_in, K_h, K_w)
    N, C_in, H, W = x.shape
    C_out, _, K_h, K_w = weight.shape

    H_out, W_out = calculate_conv_output_dim(H, W, K_h, stride, padding)

    # Pad input
    if padding > 0:
        x_padded = np.pad(x, ((0, 0), (0, 0), (padding, padding), (padding, padding)), mode='constant')
    else:
        x_padded = x

    out = np.zeros((N, C_out, H_out, W_out), dtype=np.float32)

    for n in range(N):
        for c_out in range(C_out):
            for i in range(H_out):
                h_start = i * stride
                h_end = h_start + K_h
                for j in range(W_out):
                    w_start = j * stride
                    w_end = w_start + K_w
                    patch = x_padded[n, :, h_start:h_end, w_start:w_end]
                    val = np.sum(patch * weight[c_out])
                    if bias is not None:
                        val += bias[c_out]
                    out[n, c_out, i, j] = val

    return out

def run_conv_demo():
    np.random.seed(42)
    x = np.random.randn(2, 3, 16, 16).astype(np.float32)
    w = np.random.randn(4, 3, 3, 3).astype(np.float32)
    b = np.random.randn(4).astype(np.float32)

    out_np = custom_conv2d_numpy(x, w, b, stride=1, padding=1)

    # Compare with PyTorch
    x_t = torch.from_numpy(x)
    w_t = torch.from_numpy(w)
    b_t = torch.from_numpy(b)
    out_pt = F.conv2d(x_t, w_t, b_t, stride=1, padding=1).numpy()

    max_diff = float(np.max(np.abs(out_np - out_pt)))
    print(f"2D Conv Demo: Output Shape = {out_np.shape}, Max Diff vs PyTorch = {max_diff:.6e}")
    return out_np.shape, max_diff

if __name__ == "__main__":
    run_conv_demo()
examples/test_convolutions_lib.py (1537 bytes)
import pytest
import numpy as np
import torch
import torch.nn.functional as F
from examples.convolutions_lib import calculate_conv_output_dim, custom_conv2d_numpy

def test_calculate_conv_output_dim():
    # 32x32 with K=3, S=1, P=1 -> 32x32
    assert calculate_conv_output_dim(32, 32, 3, stride=1, padding=1) == (32, 32)
    # 32x32 with K=3, S=2, P=1 -> 16x16
    assert calculate_conv_output_dim(32, 32, 3, stride=2, padding=1) == (16, 16)
    # 32x32 with K=5, S=1, P=0 -> 28x28
    assert calculate_conv_output_dim(32, 32, 5, stride=1, padding=0) == (28, 28)

def test_custom_conv2d_numpy_matches_pytorch():
    np.random.seed(42)
    x = np.random.randn(2, 3, 8, 8).astype(np.float32)
    w = np.random.randn(4, 3, 3, 3).astype(np.float32)
    b = np.random.randn(4).astype(np.float32)

    out_np = custom_conv2d_numpy(x, w, b, stride=1, padding=1)

    x_t = torch.from_numpy(x)
    w_t = torch.from_numpy(w)
    b_t = torch.from_numpy(b)
    out_pt = F.conv2d(x_t, w_t, b_t, stride=1, padding=1).numpy()

    assert np.allclose(out_np, out_pt, atol=1e-5)

def test_sobel_edge_detection_filter():
    sobel_v = np.array([
        [-1.0, 0.0, 1.0],
        [-2.0, 0.0, 2.0],
        [-1.0, 0.0, 1.0]
    ], dtype=np.float32).reshape(1, 1, 3, 3)

    img = np.zeros((1, 1, 6, 6), dtype=np.float32)
    img[:, :, :, 3:] = 1.0 # Vertical edge at col 3

    out = custom_conv2d_numpy(img, sobel_v, padding=1)
    # At column 2 and 3, horizontal gradient should be non-zero
    assert out[0, 0, 2, 2] != 0.0 or out[0, 0, 2, 3] != 0.0
metadata.yml (425 bytes)
lesson_id: D211
day: 211
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/convolutions_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/convolutions_lib.py (549 bytes)
import numpy as np
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any

def custom_conv2d_numpy(x: np.ndarray, weight: np.ndarray, bias: np.ndarray = None,
                         stride: int = 1, padding: int = 0) -> np.ndarray:
    # TODO: Implement 2D multi-channel convolution in pure NumPy
    pass

def calculate_conv_output_dim(h_in: int, w_in: int, k: int,
                              stride: int = 1, padding: int = 0) -> Tuple[int, int]:
    # TODO: Calculate output height and width using canonical formula
    pass
starter/test_convolutions_lib.py (1537 bytes)
import pytest
import numpy as np
import torch
import torch.nn.functional as F
from examples.convolutions_lib import calculate_conv_output_dim, custom_conv2d_numpy

def test_calculate_conv_output_dim():
    # 32x32 with K=3, S=1, P=1 -> 32x32
    assert calculate_conv_output_dim(32, 32, 3, stride=1, padding=1) == (32, 32)
    # 32x32 with K=3, S=2, P=1 -> 16x16
    assert calculate_conv_output_dim(32, 32, 3, stride=2, padding=1) == (16, 16)
    # 32x32 with K=5, S=1, P=0 -> 28x28
    assert calculate_conv_output_dim(32, 32, 5, stride=1, padding=0) == (28, 28)

def test_custom_conv2d_numpy_matches_pytorch():
    np.random.seed(42)
    x = np.random.randn(2, 3, 8, 8).astype(np.float32)
    w = np.random.randn(4, 3, 3, 3).astype(np.float32)
    b = np.random.randn(4).astype(np.float32)

    out_np = custom_conv2d_numpy(x, w, b, stride=1, padding=1)

    x_t = torch.from_numpy(x)
    w_t = torch.from_numpy(w)
    b_t = torch.from_numpy(b)
    out_pt = F.conv2d(x_t, w_t, b_t, stride=1, padding=1).numpy()

    assert np.allclose(out_np, out_pt, atol=1e-5)

def test_sobel_edge_detection_filter():
    sobel_v = np.array([
        [-1.0, 0.0, 1.0],
        [-2.0, 0.0, 2.0],
        [-1.0, 0.0, 1.0]
    ], dtype=np.float32).reshape(1, 1, 3, 3)

    img = np.zeros((1, 1, 6, 6), dtype=np.float32)
    img[:, :, :, 3:] = 1.0 # Vertical edge at col 3

    out = custom_conv2d_numpy(img, sobel_v, padding=1)
    # At column 2 and 3, horizontal gradient should be non-zero
    assert out[0, 0, 2, 2] != 0.0 or out[0, 0, 2, 3] != 0.0
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 211 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_convolutions_lib.py (1537 bytes)
import pytest
import numpy as np
import torch
import torch.nn.functional as F
from examples.convolutions_lib import calculate_conv_output_dim, custom_conv2d_numpy

def test_calculate_conv_output_dim():
    # 32x32 with K=3, S=1, P=1 -> 32x32
    assert calculate_conv_output_dim(32, 32, 3, stride=1, padding=1) == (32, 32)
    # 32x32 with K=3, S=2, P=1 -> 16x16
    assert calculate_conv_output_dim(32, 32, 3, stride=2, padding=1) == (16, 16)
    # 32x32 with K=5, S=1, P=0 -> 28x28
    assert calculate_conv_output_dim(32, 32, 5, stride=1, padding=0) == (28, 28)

def test_custom_conv2d_numpy_matches_pytorch():
    np.random.seed(42)
    x = np.random.randn(2, 3, 8, 8).astype(np.float32)
    w = np.random.randn(4, 3, 3, 3).astype(np.float32)
    b = np.random.randn(4).astype(np.float32)

    out_np = custom_conv2d_numpy(x, w, b, stride=1, padding=1)

    x_t = torch.from_numpy(x)
    w_t = torch.from_numpy(w)
    b_t = torch.from_numpy(b)
    out_pt = F.conv2d(x_t, w_t, b_t, stride=1, padding=1).numpy()

    assert np.allclose(out_np, out_pt, atol=1e-5)

def test_sobel_edge_detection_filter():
    sobel_v = np.array([
        [-1.0, 0.0, 1.0],
        [-2.0, 0.0, 2.0],
        [-1.0, 0.0, 1.0]
    ], dtype=np.float32).reshape(1, 1, 3, 3)

    img = np.zeros((1, 1, 6, 6), dtype=np.float32)
    img[:, :, :, 3:] = 1.0 # Vertical edge at col 3

    out = custom_conv2d_numpy(img, sobel_v, padding=1)
    # At column 2 and 3, horizontal gradient should be non-zero
    assert out[0, 0, 2, 2] != 0.0 or out[0, 0, 2, 3] != 0.0

Troubleshooting

Troubleshooting: Day 211 - Convolutions

Common Issues

  1. Output spatial dimensions off by one:
    • Cause: Forgetting to add +1 at the end of the dimension formula.
    • Fix: Always use ((H_in - K + 2*P) // S) + 1.

Security notes

Security & Privacy: Day 211 - Convolutions

Security Guidance

  • All tensor convolutions execute in local CPU memory without external network calls.