Deep Learning › Convolutional Networks and Vision › Day 212

Hands-on lab — Day 212: CNN Architectures

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/cnn_architectures_lib.py

Test

./tests/run_tests.sh

File tree

conftest.py
examples/cnn_architectures_lib.py
examples/test_cnn_architectures_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/cnn_architectures_lib.py
starter/test_cnn_architectures_lib.py
tests/run_tests.sh
tests/test_cnn_architectures_lib.py
troubleshooting.md

Lab README

Lab: Day 212 -- CNN Architectures

Lesson

Day number: 212 of 365. Course: Course05-SS01 (Deep Learning - Neural Networks). Topic: CNN Architectures and ResNet in PyTorch.

Purpose

Build and test a modular Deep Residual Network (ResNet) architecture in PyTorch. Implement BasicResidualBlock with 1x1 projection shortcuts, construct an end-to-end MiniResNet classifier with Global Average Pooling, and verify uninhibited gradient backpropagation across deep residual stages.

Learning objectives

  • Implement the BasicResidualBlock with residual addition and projection shortcuts.
  • Construct MiniResNet with stem, residual stages, and adaptive pooling.
  • Verify gradient flow through identity shortcut connections.
  • Analyze parameter counts and computational scaling in deep vision networks.

Prerequisites

  • Day 211 (Convolutions).
  • Python 3.11+ with PyTorch.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 1 GB RAM.
  • 100 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

PyTorch is free and open-source under the modified BSD license.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/cnn_architectures_lib.py: Student scaffold file.
  • examples/cnn_architectures_lib.py: Complete reference implementation.
  • tests/test_cnn_architectures_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/cnn_architectures_lib.py

What the commands do

  • Instantiates a MiniResNet model.
  • Evaluates forward pass on synthetic 32x32 image tensors.
  • Calculates parameter count and checks gradient backpropagation.

Expected output

MiniResNet Demo: Out Shape = torch.Size([2, 10]), Params = 72618

Validation steps

  1. Verify that BasicResidualBlock executes spatial downsampling correctly.
  2. Confirm that all model parameters receive non-zero gradients during backprop.
  3. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Dimension Mismatch on Addition: Verify shortcut projection layer is used when stride != 1 or in_channels != out_channels.

Security notes

All neural calculations execute locally in CPU memory without external network calls.

Extension exercises

  1. Implement a ResNet Bottleneck Block and benchmark memory usage.
  2. Integrate Squeeze-and-Excitation (SE) channel attention blocks.
  • Lesson title: CNN Architectures
  • Day number: 212 of 365
  • Lesson article: https://ai-roadmap-365.github.io/day-212-cnn-architectures
  • Lab files: everything you need is in this directory — follow “How to run” below.
  • Browse the course locally: from the repository root, this lab also appears in the course website at /labs/day-212-cnn-architectures when the site is running.

Expected output

FIELDS.md

# Expected Output Fields: Day 212

- `Out Shape`: Tensor shape of model logits output.
- `Params`: Total learnable parameter count of MiniResNet.

examples-run.txt

MiniResNet Demo: Out Shape = torch.Size([2, 10]), Params = 72618

measured-values.txt

Out Shape: torch.Size([2, 10])
Params: 72618

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 3 items

tests/test_cnn_architectures_lib.py::test_basic_residual_block_same_dim PASSED [ 33%]
tests/test_cnn_architectures_lib.py::test_basic_residual_block_downsample PASSED [ 66%]
tests/test_cnn_architectures_lib.py::test_mini_resnet_gradient_flow PASSED [100%]

============================== 3 passed in 0.28s ===============================

Source files

conftest.py (585 bytes)
"""Make this lab importable when the suite runs.

pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""

import os
import sys

LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
    if os.path.isdir(candidate) and candidate not in sys.path:
        sys.path.insert(0, candidate)
examples/cnn_architectures_lib.py (2386 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any

class BasicResidualBlock(nn.Module):
    def __init__(self, in_channels: int, out_channels: int, stride: int = 1):
        super().__init__()
        self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size=3,
                               stride=stride, padding=1, bias=False)
        self.bn1 = nn.BatchNorm2d(out_channels)
        self.relu = nn.ReLU(inplace=True)

        self.conv2 = nn.Conv2d(out_channels, out_channels, kernel_size=3,
                               stride=1, padding=1, bias=False)
        self.bn2 = nn.BatchNorm2d(out_channels)

        self.shortcut = nn.Sequential()
        if stride != 1 or in_channels != out_channels:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(out_channels)
            )

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        residual = self.shortcut(x)
        out = self.relu(self.bn1(self.conv1(x)))
        out = self.bn2(self.conv2(out))
        out += residual
        return self.relu(out)

class MiniResNet(nn.Module):
    def __init__(self, in_channels: int = 3, num_classes: int = 10):
        super().__init__()
        self.stem = nn.Sequential(
            nn.Conv2d(in_channels, 16, kernel_size=3, stride=1, padding=1, bias=False),
            nn.BatchNorm2d(16),
            nn.ReLU(inplace=True)
        )
        self.layer1 = BasicResidualBlock(16, 16, stride=1)
        self.layer2 = BasicResidualBlock(16, 32, stride=2)
        self.layer3 = BasicResidualBlock(32, 64, stride=2)

        self.gap = nn.AdaptiveAvgPool2d((1, 1))
        self.fc = nn.Linear(64, num_classes)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        x = self.stem(x)
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.gap(x)
        x = torch.flatten(x, 1)
        return self.fc(x)

def run_resnet_demo():
    torch.manual_seed(42)
    model = MiniResNet(in_channels=3, num_classes=10)
    x = torch.randn(2, 3, 32, 32)
    out = model(x)

    param_count = sum(p.numel() for p in model.parameters())
    print(f"MiniResNet Demo: Out Shape = {out.shape}, Params = {param_count}")
    return out.shape, param_count

if __name__ == "__main__":
    run_resnet_demo()
examples/test_cnn_architectures_lib.py (1017 bytes)
import pytest
import torch
from examples.cnn_architectures_lib import BasicResidualBlock, MiniResNet

def test_basic_residual_block_same_dim():
    block = BasicResidualBlock(in_channels=16, out_channels=16, stride=1)
    x = torch.randn(2, 16, 8, 8)
    out = block(x)
    assert out.shape == (2, 16, 8, 8)

def test_basic_residual_block_downsample():
    block = BasicResidualBlock(in_channels=16, out_channels=32, stride=2)
    x = torch.randn(2, 16, 8, 8)
    out = block(x)
    assert out.shape == (2, 32, 4, 4)

def test_mini_resnet_gradient_flow():
    model = MiniResNet(in_channels=3, num_classes=5)
    x = torch.randn(4, 3, 32, 32)
    y = torch.randint(0, 5, (4,))

    logits = model(x)
    loss = torch.nn.CrossEntropyLoss()(logits, y)
    loss.backward()

    # Verify every weight tensor in the stem and residual layers received gradients
    for name, p in model.named_parameters():
        if p.requires_grad:
            assert p.grad is not None
            assert torch.norm(p.grad).item() > 0.0
metadata.yml (430 bytes)
lesson_id: D212
day: 212
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/cnn_architectures_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (27 bytes)
torch>=2.2.0
pytest>=8.0.0
starter/cnn_architectures_lib.py (643 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any

class BasicResidualBlock(nn.Module):
    def __init__(self, in_channels: int, out_channels: int, stride: int = 1):
        super().__init__()
        # TODO: Implement Residual Block with 1x1 projection shortcut
        pass

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        pass

class MiniResNet(nn.Module):
    def __init__(self, num_classes: int = 10):
        super().__init__()
        # TODO: Implement MiniResNet architecture with stem, stages, GAP, and classifier
        pass

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        pass
starter/test_cnn_architectures_lib.py (1017 bytes)
import pytest
import torch
from examples.cnn_architectures_lib import BasicResidualBlock, MiniResNet

def test_basic_residual_block_same_dim():
    block = BasicResidualBlock(in_channels=16, out_channels=16, stride=1)
    x = torch.randn(2, 16, 8, 8)
    out = block(x)
    assert out.shape == (2, 16, 8, 8)

def test_basic_residual_block_downsample():
    block = BasicResidualBlock(in_channels=16, out_channels=32, stride=2)
    x = torch.randn(2, 16, 8, 8)
    out = block(x)
    assert out.shape == (2, 32, 4, 4)

def test_mini_resnet_gradient_flow():
    model = MiniResNet(in_channels=3, num_classes=5)
    x = torch.randn(4, 3, 32, 32)
    y = torch.randint(0, 5, (4,))

    logits = model(x)
    loss = torch.nn.CrossEntropyLoss()(logits, y)
    loss.backward()

    # Verify every weight tensor in the stem and residual layers received gradients
    for name, p in model.named_parameters():
        if p.requires_grad:
            assert p.grad is not None
            assert torch.norm(p.grad).item() > 0.0
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 212 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_cnn_architectures_lib.py (1017 bytes)
import pytest
import torch
from examples.cnn_architectures_lib import BasicResidualBlock, MiniResNet

def test_basic_residual_block_same_dim():
    block = BasicResidualBlock(in_channels=16, out_channels=16, stride=1)
    x = torch.randn(2, 16, 8, 8)
    out = block(x)
    assert out.shape == (2, 16, 8, 8)

def test_basic_residual_block_downsample():
    block = BasicResidualBlock(in_channels=16, out_channels=32, stride=2)
    x = torch.randn(2, 16, 8, 8)
    out = block(x)
    assert out.shape == (2, 32, 4, 4)

def test_mini_resnet_gradient_flow():
    model = MiniResNet(in_channels=3, num_classes=5)
    x = torch.randn(4, 3, 32, 32)
    y = torch.randint(0, 5, (4,))

    logits = model(x)
    loss = torch.nn.CrossEntropyLoss()(logits, y)
    loss.backward()

    # Verify every weight tensor in the stem and residual layers received gradients
    for name, p in model.named_parameters():
        if p.requires_grad:
            assert p.grad is not None
            assert torch.norm(p.grad).item() > 0.0

Troubleshooting

Troubleshooting: Day 212 - CNN Architectures

Common Issues

  1. Shape mismatch at out += residual:
    • Cause: Input tensor channels do not match output channels.
    • Fix: Use a 1x1 conv projection in the shortcut path.

Security notes

Security & Privacy: Day 212 - CNN Architectures

Security Guidance

  • All vision calculations run locally in memory without external telemetric logging.