Deep Learning › Convolutional Networks and Vision › Day 213
Hands-on lab — Day 213: Transfer Learning
- ← Back to the Day 213 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/deep-learning/day-213-transfer-learning/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/transfer_learning_lib.py Test
./tests/run_tests.sh File tree
conftest.py examples/test_transfer_learning_lib.py examples/transfer_learning_lib.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/starter-run.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/test_transfer_learning_lib.py starter/transfer_learning_lib.py tests/run_tests.sh tests/test_transfer_learning_lib.py troubleshooting.md
Lab README
Lab: Day 213 -- Transfer Learning
Lesson
Day number: 213 of 365. Course: Course05-SS01 (Deep Learning - Neural Networks). Topic: Transfer Learning and Fine-Tuning in PyTorch.
Purpose
Build and test a modular Transfer Learning architecture in PyTorch. Implement parameter freezing routines with requires_grad, replace classification heads for custom target datasets, configure differential layer-wise learning rates, and verify that frozen backbones receive zero gradient updates during backward propagation.
Learning objectives
- Freeze and unfreeze parameter subsets in PyTorch vision models.
- Replace fully connected heads to support arbitrary target classes.
- Verify gradient isolation during backpropagation.
- Configure multi-group differential learning rate optimizers.
Prerequisites
- Day 212 (CNN Architectures).
- Python 3.11+ with PyTorch.
Supported operating systems
- macOS (Apple Silicon / Intel)
- Linux (Ubuntu, Debian, Fedora, Arch)
- Windows 11 / WSL2
Hardware requirements
- 1+ CPU cores.
- 1 GB RAM.
- 100 MB disk space.
Required software
- Python 3.11 or newer.
- pip package manager.
- virtualenv or venv module.
Free and open-source options
PyTorch is free and open-source under the modified BSD license.
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/transfer_learning_lib.py: Student scaffold file.examples/transfer_learning_lib.py: Complete reference implementation.tests/test_transfer_learning_lib.py: Pytest automated validation suite.expected-output/: Verified output logs and baseline values.
How to run
Execute the reference demonstration script:
python3 examples/transfer_learning_lib.py
What the commands do
- Instantiates a TransferVisionModel with a frozen backbone.
- Evaluates forward pass and counts frozen vs trainable parameters.
- Validates gradient isolation and differential optimizer grouping.
Expected output
Transfer Demo: Out = torch.Size([2, 5]), Frozen = 5120, Trainable = 613
Validation steps
- Verify that
requires_grad=Falseparameters receiveNonegradients during backprop. - Confirm that newly added classification heads receive valid gradients.
- Ensure all unit test assertions pass.
Tests
Run the test runner script:
./tests/run_tests.sh
Cleanup
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +
Troubleshooting
- Backbone Receives Gradients: Ensure
requires_grad = Falseis set before invoking the optimizer.
Security notes
All neural models execute in local system memory without external telemetry.
Extension exercises
- Implement embedding caching: extract backbone feature vectors to disk and train a linear classifier in 1 second.
- Implement gradual unfreezing across successive training epochs.
Navigation
- Lesson title: Transfer Learning
- Day number: 213 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-213-transfer-learning
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-213-transfer-learningwhen the site is running.
Expected output
FIELDS.md
# Expected Output Fields: Day 213
- `Out`: Output logits shape.
- `Frozen`: Total frozen parameters in backbone.
- `Trainable`: Total trainable parameters in new head.
examples-run.txt
Transfer Demo: Out = torch.Size([2, 5]), Frozen = 5120, Trainable = 613
measured-values.txt
Out: torch.Size([2, 5])
Frozen: 5120
Trainable: 613
starter-run.txt
Starter scaffold executed. Ready for student implementation.
test-run.txt
============================= test session starts ==============================
collected 3 items
tests/test_transfer_learning_lib.py::test_frozen_backbone_parameters PASSED [ 33%]
tests/test_transfer_learning_lib.py::test_gradient_isolation PASSED [ 66%]
tests/test_transfer_learning_lib.py::test_differential_optimizer_groups PASSED [100%]
============================== 3 passed in 0.29s ===============================
Source files
conftest.py (585 bytes)
"""Make this lab importable when the suite runs.
pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""
import os
import sys
LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
if os.path.isdir(candidate) and candidate not in sys.path:
sys.path.insert(0, candidate)
examples/test_transfer_learning_lib.py (1315 bytes)
import pytest
import torch
from examples.transfer_learning_lib import build_transfer_learning_model, configure_differential_optimizer
def test_frozen_backbone_parameters():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=True)
# Check that backbone is frozen
for p in model.backbone.parameters():
assert p.requires_grad is False
# Check that head is trainable
for p in model.fc.parameters():
assert p.requires_grad is True
def test_gradient_isolation():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=True)
x = torch.randn(4, 3, 32, 32)
y = torch.randint(0, 5, (4,))
logits = model(x)
loss = torch.nn.CrossEntropyLoss()(logits, y)
loss.backward()
# Backbone weights must have None grad
for p in model.backbone.parameters():
assert p.grad is None
# Head weights must have non-zero grad
for p in model.fc.parameters():
assert p.grad is not None
def test_differential_optimizer_groups():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=False)
opt = configure_differential_optimizer(model, head_lr=1e-3, backbone_lr=1e-5)
assert len(opt.param_groups) == 2
assert opt.param_groups[0]["lr"] == 1e-5
assert opt.param_groups[1]["lr"] == 1e-3
examples/transfer_learning_lib.py (2503 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any, List
class SimpleBackbone(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(3, 16, kernel_size=3, stride=1, padding=1, bias=False)
self.bn1 = nn.BatchNorm2d(16)
self.relu = nn.ReLU(inplace=True)
self.conv2 = nn.Conv2d(16, 32, kernel_size=3, stride=2, padding=1, bias=False)
self.bn2 = nn.BatchNorm2d(32)
self.gap = nn.AdaptiveAvgPool2d((1, 1))
def forward(self, x: torch.Tensor) -> torch.Tensor:
x = self.relu(self.bn1(self.conv1(x)))
x = self.relu(self.bn2(self.conv2(x)))
x = self.gap(x)
return torch.flatten(x, 1)
class TransferVisionModel(nn.Module):
def __init__(self, num_classes: int = 5, freeze_backbone: bool = True):
super().__init__()
self.backbone = SimpleBackbone()
if freeze_backbone:
for param in self.backbone.parameters():
param.requires_grad = False
self.fc = nn.Sequential(
nn.Linear(32, 16),
nn.ReLU(inplace=True),
nn.Dropout(p=0.2),
nn.Linear(16, num_classes)
)
def forward(self, x: torch.Tensor) -> torch.Tensor:
feat = self.backbone(x)
return self.fc(feat)
def build_transfer_learning_model(num_classes: int = 5, freeze_backbone: bool = True) -> nn.Module:
return TransferVisionModel(num_classes=num_classes, freeze_backbone=freeze_backbone)
def configure_differential_optimizer(model: TransferVisionModel, head_lr: float = 1e-3,
backbone_lr: float = 1e-5) -> torch.optim.Optimizer:
optimizer = torch.optim.AdamW([
{"params": [p for p in model.backbone.parameters() if p.requires_grad], "lr": backbone_lr},
{"params": model.fc.parameters(), "lr": head_lr}
], weight_decay=1e-2)
return optimizer
def run_transfer_demo():
torch.manual_seed(42)
model = build_transfer_learning_model(num_classes=5, freeze_backbone=True)
x = torch.randn(2, 3, 32, 32)
out = model(x)
frozen_params = sum(p.numel() for p in model.backbone.parameters() if not p.requires_grad)
trainable_params = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"Transfer Demo: Out = {out.shape}, Frozen = {frozen_params}, Trainable = {trainable_params}")
return out.shape, frozen_params, trainable_params
if __name__ == "__main__":
run_transfer_demo()
metadata.yml (430 bytes)
lesson_id: D213
day: 213
kind: lab
languages:
- python
setup_commands:
- 'pip install -r requirements/requirements.txt'
run_commands:
- 'python3 examples/transfer_learning_lib.py'
test_commands:
- './tests/run_tests.sh'
cleanup_commands:
- 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (27 bytes)
torch>=2.2.0
pytest>=8.0.0
starter/test_transfer_learning_lib.py (1315 bytes)
import pytest
import torch
from examples.transfer_learning_lib import build_transfer_learning_model, configure_differential_optimizer
def test_frozen_backbone_parameters():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=True)
# Check that backbone is frozen
for p in model.backbone.parameters():
assert p.requires_grad is False
# Check that head is trainable
for p in model.fc.parameters():
assert p.requires_grad is True
def test_gradient_isolation():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=True)
x = torch.randn(4, 3, 32, 32)
y = torch.randint(0, 5, (4,))
logits = model(x)
loss = torch.nn.CrossEntropyLoss()(logits, y)
loss.backward()
# Backbone weights must have None grad
for p in model.backbone.parameters():
assert p.grad is None
# Head weights must have non-zero grad
for p in model.fc.parameters():
assert p.grad is not None
def test_differential_optimizer_groups():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=False)
opt = configure_differential_optimizer(model, head_lr=1e-3, backbone_lr=1e-5)
assert len(opt.param_groups) == 2
assert opt.param_groups[0]["lr"] == 1e-5
assert opt.param_groups[1]["lr"] == 1e-3
starter/transfer_learning_lib.py (529 bytes)
import torch
import torch.nn as nn
from typing import Tuple, Dict, Any, List
def build_transfer_learning_model(num_classes: int = 5, freeze_backbone: bool = True) -> nn.Module:
# TODO: Build modular model with frozen backbone and custom classifier head
pass
def configure_differential_optimizer(model: nn.Module, head_lr: float = 1e-3,
backbone_lr: float = 1e-5) -> torch.optim.Optimizer:
# TODO: Configure PyTorch optimizer with differential layer-wise learning rates
pass
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 213 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_transfer_learning_lib.py (1315 bytes)
import pytest
import torch
from examples.transfer_learning_lib import build_transfer_learning_model, configure_differential_optimizer
def test_frozen_backbone_parameters():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=True)
# Check that backbone is frozen
for p in model.backbone.parameters():
assert p.requires_grad is False
# Check that head is trainable
for p in model.fc.parameters():
assert p.requires_grad is True
def test_gradient_isolation():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=True)
x = torch.randn(4, 3, 32, 32)
y = torch.randint(0, 5, (4,))
logits = model(x)
loss = torch.nn.CrossEntropyLoss()(logits, y)
loss.backward()
# Backbone weights must have None grad
for p in model.backbone.parameters():
assert p.grad is None
# Head weights must have non-zero grad
for p in model.fc.parameters():
assert p.grad is not None
def test_differential_optimizer_groups():
model = build_transfer_learning_model(num_classes=5, freeze_backbone=False)
opt = configure_differential_optimizer(model, head_lr=1e-3, backbone_lr=1e-5)
assert len(opt.param_groups) == 2
assert opt.param_groups[0]["lr"] == 1e-5
assert opt.param_groups[1]["lr"] == 1e-3
Troubleshooting
Troubleshooting: Day 213 - Transfer Learning
Common Issues
- Backbone weights changing when frozen:
- Cause: Passing frozen parameters to the optimizer without checking
p.requires_grad. - Fix: Filter optimizer parameters:
[p for p in model.parameters() if p.requires_grad].
- Cause: Passing frozen parameters to the optimizer without checking
Security notes
Security & Privacy: Day 213 - Transfer Learning
Security Guidance
- Always verify SHA-256 integrity when downloading pretrained vision weights.