Deep Learning › Transformers › Day 230
Hands-on lab — Day 230: Hugging Face Transformers in Practice
- ← Back to the Day 230 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/deep-learning/day-230-hugging-face-transformers-in-practice/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/hugging_face_transformers_in_practice_lib.py Test
./tests/run_tests.sh File tree
conftest.py examples/hugging_face_transformers_in_practice_lib.py examples/test_hugging_face_transformers_in_practice_lib.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/starter-run.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/hugging_face_transformers_in_practice_lib.py starter/test_hugging_face_transformers_in_practice_lib.py tests/run_tests.sh tests/test_hugging_face_transformers_in_practice_lib.py troubleshooting.md
Lab README
Lab: Day 230 -- Hugging Face Transformers in Practice
Lesson
Day number: 230 of 365. Course: Course05-SS02 (Deep Learning - Sequence Models and Transformers). Topic: Hugging Face Transformers, Dynamic Padding, and Metrics.
Purpose
Build and test a modular Hugging Face data pipeline and evaluation suite in PyTorch. Implement dynamic batch padding collation (DataCollatorWithPadding), construct attention masks, and build the metric compute engine (Accuracy, Precision, Recall, F1).
Learning objectives
- Implement dynamic batch padding collation with
DataCollatorWithPadding. - Generate accurate binary attention masks for padded sequences.
- Implement classification evaluation metrics (Accuracy, Precision, Recall, F1).
- Verify dimensional compatibility across batched tensor pipelines.
Prerequisites
- Day 229 (Decoder Models: The GPT Family).
- Python 3.11+ with PyTorch and NumPy.
Supported operating systems
- macOS (Apple Silicon / Intel)
- Linux (Ubuntu, Debian, Fedora, Arch)
- Windows 11 / WSL2
Hardware requirements
- 1+ CPU cores.
- 1 GB RAM.
- 100 MB disk space.
Required software
- Python 3.11 or newer.
- pip package manager.
- virtualenv or venv module.
Free and open-source options
PyTorch is open-source software maintained by the Linux Foundation under a modified BSD license.
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/hugging_face_transformers_in_practice_lib.py: Student scaffold file.examples/hugging_face_transformers_in_practice_lib.py: Complete reference implementation.tests/test_hugging_face_transformers_in_practice_lib.py: Pytest automated validation suite.expected-output/: Verified output logs and baseline values.
How to run
Execute the reference demonstration script:
python3 examples/hugging_face_transformers_in_practice_lib.py
What the commands do
- Evaluates dynamic batch collation across variable length sequences.
- Computes statistical accuracy and F1 metrics.
- Runs unit test assertions.
Expected output
HF Demo: Collated Batch Shape = torch.Size([2, 5])
Validation steps
- Verify
MockDataCollatorWithPaddingpads batches strictly to the longest sequence in the batch. - Confirm
attention_maskcontains1for real tokens and0for pad tokens. - Ensure all unit test assertions pass.
Tests
Run the test runner script:
./tests/run_tests.sh
Cleanup
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +
Troubleshooting
- Attention mask shape mismatch: Ensure attention mask has identical shape to
input_ids.
Security notes
All data collation computations execute locally in process memory.
Extension exercises
- Implement a Multi-Label F1 metric calculator.
- Implement a Learning Rate Warmup schedule generator.
Navigation
- Lesson title: Hugging Face Transformers in Practice
- Day number: 230 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-230-hugging-face-transformers-in-practice
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-230-hugging-face-transformers-in-practicewhen the site is running.
Expected output
FIELDS.md
# Expected Output Fields: Day 230
- `Collated Batch Shape`: Tensor shape of the dynamically padded input batch.
examples-run.txt
HF Demo: Collated Batch Shape = torch.Size([2, 5])
measured-values.txt
Collated Batch Shape: torch.Size([2, 5])
starter-run.txt
Starter scaffold executed. Ready for student implementation.
test-run.txt
============================= test session starts ==============================
collected 2 items
tests/test_hugging_face_transformers_in_practice_lib.py::test_dynamic_padding_collator PASSED [ 50%]
tests/test_hugging_face_transformers_in_practice_lib.py::test_metric_computation PASSED [100%]
============================== 2 passed in 0.43s ===============================
Source files
conftest.py (585 bytes)
"""Make this lab importable when the suite runs.
pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""
import os
import sys
LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
if os.path.isdir(candidate) and candidate not in sys.path:
sys.path.insert(0, candidate)
examples/hugging_face_transformers_in_practice_lib.py (2214 bytes)
import torch
import torch.nn as nn
import numpy as np
from typing import Dict, List, Any
class MockDataCollatorWithPadding:
def __init__(self, pad_token_id: int = 0):
self.pad_token_id = pad_token_id
def __call__(self, batch: List[Dict[str, Any]]) -> Dict[str, torch.Tensor]:
max_len = max(len(item["input_ids"]) for item in batch)
batch_input_ids = []
batch_attention_mask = []
batch_labels = []
for item in batch:
ids = item["input_ids"]
pad_len = max_len - len(ids)
padded_ids = ids + [self.pad_token_id] * pad_len
attn_mask = [1] * len(ids) + [0] * pad_len
batch_input_ids.append(padded_ids)
batch_attention_mask.append(attn_mask)
if "label" in item:
batch_labels.append(item["label"])
res = {
"input_ids": torch.tensor(batch_input_ids, dtype=torch.long),
"attention_mask": torch.tensor(batch_attention_mask, dtype=torch.long)
}
if batch_labels:
res["labels"] = torch.tensor(batch_labels, dtype=torch.long)
return res
def compute_classification_metrics(predictions: np.ndarray, labels: np.ndarray) -> Dict[str, float]:
accuracy = float(np.mean(predictions == labels))
tp = float(np.sum((predictions == 1) & (labels == 1)))
fp = float(np.sum((predictions == 1) & (labels == 0)))
fn = float(np.sum((predictions == 0) & (labels == 1)))
precision = tp / (tp + fp + 1e-8)
recall = tp / (tp + fn + 1e-8)
f1 = 2 * (precision * recall) / (precision + recall + 1e-8)
return {
"accuracy": round(accuracy, 4),
"precision": round(precision, 4),
"recall": round(recall, 4),
"f1": round(f1, 4)
}
def run_hf_demo():
collator = MockDataCollatorWithPadding(pad_token_id=0)
sample_batch = [
{"input_ids": [101, 2054, 102], "label": 1},
{"input_ids": [101, 1037, 3899, 2003, 102], "label": 0}
]
batch = collator(sample_batch)
print(f"HF Demo: Collated Batch Shape = {batch['input_ids'].shape}")
return batch
if __name__ == "__main__":
run_hf_demo()
examples/test_hugging_face_transformers_in_practice_lib.py (858 bytes)
import pytest
import torch
import numpy as np
from examples.hugging_face_transformers_in_practice_lib import MockDataCollatorWithPadding, compute_classification_metrics
def test_dynamic_padding_collator():
collator = MockDataCollatorWithPadding(pad_token_id=0)
batch_data = [
{"input_ids": [1, 2, 3], "label": 1},
{"input_ids": [4, 5, 6, 7, 8], "label": 0}
]
res = collator(batch_data)
assert res["input_ids"].shape == (2, 5)
assert res["attention_mask"].shape == (2, 5)
assert res["attention_mask"][0].tolist() == [1, 1, 1, 0, 0]
assert res["labels"].shape == (2,)
def test_metric_computation():
preds = np.array([1, 0, 1, 1, 0])
labels = np.array([1, 0, 1, 0, 0])
metrics = compute_classification_metrics(preds, labels)
assert metrics["accuracy"] == 0.8
assert metrics["f1"] > 0.7
metadata.yml (450 bytes)
lesson_id: D230
day: 230
kind: lab
languages:
- python
setup_commands:
- 'pip install -r requirements/requirements.txt'
run_commands:
- 'python3 examples/hugging_face_transformers_in_practice_lib.py'
test_commands:
- './tests/run_tests.sh'
cleanup_commands:
- 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/hugging_face_transformers_in_practice_lib.py (566 bytes)
import torch
import torch.nn as nn
import numpy as np
from typing import Dict, List, Any
class MockDataCollatorWithPadding:
def __init__(self, pad_token_id: int = 0):
self.pad_token_id = pad_token_id
def __call__(self, batch: List[Dict[str, Any]]) -> Dict[str, torch.Tensor]:
# TODO: Implement dynamic batch padding across input_ids and attention_mask
pass
def compute_classification_metrics(predictions: np.ndarray, labels: np.ndarray) -> Dict[str, float]:
# TODO: Calculate accuracy, precision, recall, and f1 score
pass
starter/test_hugging_face_transformers_in_practice_lib.py (858 bytes)
import pytest
import torch
import numpy as np
from examples.hugging_face_transformers_in_practice_lib import MockDataCollatorWithPadding, compute_classification_metrics
def test_dynamic_padding_collator():
collator = MockDataCollatorWithPadding(pad_token_id=0)
batch_data = [
{"input_ids": [1, 2, 3], "label": 1},
{"input_ids": [4, 5, 6, 7, 8], "label": 0}
]
res = collator(batch_data)
assert res["input_ids"].shape == (2, 5)
assert res["attention_mask"].shape == (2, 5)
assert res["attention_mask"][0].tolist() == [1, 1, 1, 0, 0]
assert res["labels"].shape == (2,)
def test_metric_computation():
preds = np.array([1, 0, 1, 1, 0])
labels = np.array([1, 0, 1, 0, 0])
metrics = compute_classification_metrics(preds, labels)
assert metrics["accuracy"] == 0.8
assert metrics["f1"] > 0.7
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 230 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_hugging_face_transformers_in_practice_lib.py (858 bytes)
import pytest
import torch
import numpy as np
from examples.hugging_face_transformers_in_practice_lib import MockDataCollatorWithPadding, compute_classification_metrics
def test_dynamic_padding_collator():
collator = MockDataCollatorWithPadding(pad_token_id=0)
batch_data = [
{"input_ids": [1, 2, 3], "label": 1},
{"input_ids": [4, 5, 6, 7, 8], "label": 0}
]
res = collator(batch_data)
assert res["input_ids"].shape == (2, 5)
assert res["attention_mask"].shape == (2, 5)
assert res["attention_mask"][0].tolist() == [1, 1, 1, 0, 0]
assert res["labels"].shape == (2,)
def test_metric_computation():
preds = np.array([1, 0, 1, 1, 0])
labels = np.array([1, 0, 1, 0, 0])
metrics = compute_classification_metrics(preds, labels)
assert metrics["accuracy"] == 0.8
assert metrics["f1"] > 0.7
Troubleshooting
Troubleshooting: Day 230 - Hugging Face Transformers in Practice
Common Issues
- Pad token id confusion:
- Cause: Using default pad token 0 when model expects different id.
- Fix: Use
tokenizer.pad_token_id.
Security notes
Security & Privacy: Day 230 - Hugging Face Transformers in Practice
Security Guidance
- Always serialize and load models in
safetensorsformat rather than pickle files.