Deep Learning › Transformers › Day 230

Hands-on lab — Day 230: Hugging Face Transformers in Practice

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/hugging_face_transformers_in_practice_lib.py

Test

./tests/run_tests.sh

File tree

conftest.py
examples/hugging_face_transformers_in_practice_lib.py
examples/test_hugging_face_transformers_in_practice_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/hugging_face_transformers_in_practice_lib.py
starter/test_hugging_face_transformers_in_practice_lib.py
tests/run_tests.sh
tests/test_hugging_face_transformers_in_practice_lib.py
troubleshooting.md

Lab README

Lab: Day 230 -- Hugging Face Transformers in Practice

Lesson

Day number: 230 of 365. Course: Course05-SS02 (Deep Learning - Sequence Models and Transformers). Topic: Hugging Face Transformers, Dynamic Padding, and Metrics.

Purpose

Build and test a modular Hugging Face data pipeline and evaluation suite in PyTorch. Implement dynamic batch padding collation (DataCollatorWithPadding), construct attention masks, and build the metric compute engine (Accuracy, Precision, Recall, F1).

Learning objectives

  • Implement dynamic batch padding collation with DataCollatorWithPadding.
  • Generate accurate binary attention masks for padded sequences.
  • Implement classification evaluation metrics (Accuracy, Precision, Recall, F1).
  • Verify dimensional compatibility across batched tensor pipelines.

Prerequisites

  • Day 229 (Decoder Models: The GPT Family).
  • Python 3.11+ with PyTorch and NumPy.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 1 GB RAM.
  • 100 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

PyTorch is open-source software maintained by the Linux Foundation under a modified BSD license.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/hugging_face_transformers_in_practice_lib.py: Student scaffold file.
  • examples/hugging_face_transformers_in_practice_lib.py: Complete reference implementation.
  • tests/test_hugging_face_transformers_in_practice_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/hugging_face_transformers_in_practice_lib.py

What the commands do

  • Evaluates dynamic batch collation across variable length sequences.
  • Computes statistical accuracy and F1 metrics.
  • Runs unit test assertions.

Expected output

HF Demo: Collated Batch Shape = torch.Size([2, 5])

Validation steps

  1. Verify MockDataCollatorWithPadding pads batches strictly to the longest sequence in the batch.
  2. Confirm attention_mask contains 1 for real tokens and 0 for pad tokens.
  3. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Attention mask shape mismatch: Ensure attention mask has identical shape to input_ids.

Security notes

All data collation computations execute locally in process memory.

Extension exercises

  1. Implement a Multi-Label F1 metric calculator.
  2. Implement a Learning Rate Warmup schedule generator.

Expected output

FIELDS.md

# Expected Output Fields: Day 230

- `Collated Batch Shape`: Tensor shape of the dynamically padded input batch.

examples-run.txt

HF Demo: Collated Batch Shape = torch.Size([2, 5])

measured-values.txt

Collated Batch Shape: torch.Size([2, 5])

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 2 items

tests/test_hugging_face_transformers_in_practice_lib.py::test_dynamic_padding_collator PASSED [ 50%]
tests/test_hugging_face_transformers_in_practice_lib.py::test_metric_computation PASSED        [100%]

============================== 2 passed in 0.43s ===============================

Source files

conftest.py (585 bytes)
"""Make this lab importable when the suite runs.

pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""

import os
import sys

LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
    if os.path.isdir(candidate) and candidate not in sys.path:
        sys.path.insert(0, candidate)
examples/hugging_face_transformers_in_practice_lib.py (2214 bytes)
import torch
import torch.nn as nn
import numpy as np
from typing import Dict, List, Any

class MockDataCollatorWithPadding:
    def __init__(self, pad_token_id: int = 0):
        self.pad_token_id = pad_token_id

    def __call__(self, batch: List[Dict[str, Any]]) -> Dict[str, torch.Tensor]:
        max_len = max(len(item["input_ids"]) for item in batch)
        
        batch_input_ids = []
        batch_attention_mask = []
        batch_labels = []

        for item in batch:
            ids = item["input_ids"]
            pad_len = max_len - len(ids)
            
            padded_ids = ids + [self.pad_token_id] * pad_len
            attn_mask = [1] * len(ids) + [0] * pad_len
            
            batch_input_ids.append(padded_ids)
            batch_attention_mask.append(attn_mask)
            if "label" in item:
                batch_labels.append(item["label"])

        res = {
            "input_ids": torch.tensor(batch_input_ids, dtype=torch.long),
            "attention_mask": torch.tensor(batch_attention_mask, dtype=torch.long)
        }
        if batch_labels:
            res["labels"] = torch.tensor(batch_labels, dtype=torch.long)
        return res

def compute_classification_metrics(predictions: np.ndarray, labels: np.ndarray) -> Dict[str, float]:
    accuracy = float(np.mean(predictions == labels))
    tp = float(np.sum((predictions == 1) & (labels == 1)))
    fp = float(np.sum((predictions == 1) & (labels == 0)))
    fn = float(np.sum((predictions == 0) & (labels == 1)))

    precision = tp / (tp + fp + 1e-8)
    recall = tp / (tp + fn + 1e-8)
    f1 = 2 * (precision * recall) / (precision + recall + 1e-8)

    return {
        "accuracy": round(accuracy, 4),
        "precision": round(precision, 4),
        "recall": round(recall, 4),
        "f1": round(f1, 4)
    }

def run_hf_demo():
    collator = MockDataCollatorWithPadding(pad_token_id=0)
    sample_batch = [
        {"input_ids": [101, 2054, 102], "label": 1},
        {"input_ids": [101, 1037, 3899, 2003, 102], "label": 0}
    ]
    batch = collator(sample_batch)
    print(f"HF Demo: Collated Batch Shape = {batch['input_ids'].shape}")
    return batch

if __name__ == "__main__":
    run_hf_demo()
examples/test_hugging_face_transformers_in_practice_lib.py (858 bytes)
import pytest
import torch
import numpy as np
from examples.hugging_face_transformers_in_practice_lib import MockDataCollatorWithPadding, compute_classification_metrics

def test_dynamic_padding_collator():
    collator = MockDataCollatorWithPadding(pad_token_id=0)
    batch_data = [
        {"input_ids": [1, 2, 3], "label": 1},
        {"input_ids": [4, 5, 6, 7, 8], "label": 0}
    ]
    res = collator(batch_data)
    assert res["input_ids"].shape == (2, 5)
    assert res["attention_mask"].shape == (2, 5)
    assert res["attention_mask"][0].tolist() == [1, 1, 1, 0, 0]
    assert res["labels"].shape == (2,)

def test_metric_computation():
    preds = np.array([1, 0, 1, 1, 0])
    labels = np.array([1, 0, 1, 0, 0])
    metrics = compute_classification_metrics(preds, labels)
    
    assert metrics["accuracy"] == 0.8
    assert metrics["f1"] > 0.7
metadata.yml (450 bytes)
lesson_id: D230
day: 230
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/hugging_face_transformers_in_practice_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/hugging_face_transformers_in_practice_lib.py (566 bytes)
import torch
import torch.nn as nn
import numpy as np
from typing import Dict, List, Any

class MockDataCollatorWithPadding:
    def __init__(self, pad_token_id: int = 0):
        self.pad_token_id = pad_token_id

    def __call__(self, batch: List[Dict[str, Any]]) -> Dict[str, torch.Tensor]:
        # TODO: Implement dynamic batch padding across input_ids and attention_mask
        pass

def compute_classification_metrics(predictions: np.ndarray, labels: np.ndarray) -> Dict[str, float]:
    # TODO: Calculate accuracy, precision, recall, and f1 score
    pass
starter/test_hugging_face_transformers_in_practice_lib.py (858 bytes)
import pytest
import torch
import numpy as np
from examples.hugging_face_transformers_in_practice_lib import MockDataCollatorWithPadding, compute_classification_metrics

def test_dynamic_padding_collator():
    collator = MockDataCollatorWithPadding(pad_token_id=0)
    batch_data = [
        {"input_ids": [1, 2, 3], "label": 1},
        {"input_ids": [4, 5, 6, 7, 8], "label": 0}
    ]
    res = collator(batch_data)
    assert res["input_ids"].shape == (2, 5)
    assert res["attention_mask"].shape == (2, 5)
    assert res["attention_mask"][0].tolist() == [1, 1, 1, 0, 0]
    assert res["labels"].shape == (2,)

def test_metric_computation():
    preds = np.array([1, 0, 1, 1, 0])
    labels = np.array([1, 0, 1, 0, 0])
    metrics = compute_classification_metrics(preds, labels)
    
    assert metrics["accuracy"] == 0.8
    assert metrics["f1"] > 0.7
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 230 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_hugging_face_transformers_in_practice_lib.py (858 bytes)
import pytest
import torch
import numpy as np
from examples.hugging_face_transformers_in_practice_lib import MockDataCollatorWithPadding, compute_classification_metrics

def test_dynamic_padding_collator():
    collator = MockDataCollatorWithPadding(pad_token_id=0)
    batch_data = [
        {"input_ids": [1, 2, 3], "label": 1},
        {"input_ids": [4, 5, 6, 7, 8], "label": 0}
    ]
    res = collator(batch_data)
    assert res["input_ids"].shape == (2, 5)
    assert res["attention_mask"].shape == (2, 5)
    assert res["attention_mask"][0].tolist() == [1, 1, 1, 0, 0]
    assert res["labels"].shape == (2,)

def test_metric_computation():
    preds = np.array([1, 0, 1, 1, 0])
    labels = np.array([1, 0, 1, 0, 0])
    metrics = compute_classification_metrics(preds, labels)
    
    assert metrics["accuracy"] == 0.8
    assert metrics["f1"] > 0.7

Troubleshooting

Troubleshooting: Day 230 - Hugging Face Transformers in Practice

Common Issues

  1. Pad token id confusion:
    • Cause: Using default pad token 0 when model expects different id.
    • Fix: Use tokenizer.pad_token_id.

Security notes

Security & Privacy: Day 230 - Hugging Face Transformers in Practice

Security Guidance

  • Always serialize and load models in safetensors format rather than pickle files.