Deep Learning › Sequences and Text › Day 223

Hands-on lab — Day 223: Text Classification with Embeddings

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/text_classification_with_embeddings_lib.py

Test

./tests/run_tests.sh

File tree

conftest.py
examples/test_text_classification_with_embeddings_lib.py
examples/text_classification_with_embeddings_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/test_text_classification_with_embeddings_lib.py
starter/text_classification_with_embeddings_lib.py
tests/run_tests.sh
tests/test_text_classification_with_embeddings_lib.py
troubleshooting.md

Lab README

Lab: Day 223 -- Text Classification with Embeddings

Lesson

Day number: 223 of 365. Course: Course05-SS02 (Deep Learning - Sequence Models and Transformers). Topic: Neural Text Classification Architectures.

Purpose

Build and test modular neural text classification architectures in PyTorch. Implement the fast EmbeddingBagClassifier with fused average pooling, build the Yoon Kim TextCNN multi-scale 1D convolution model with max-over-time pooling, and verify classification logits across variable-length text batches.

Learning objectives

  • Implement nn.EmbeddingBag document representations with fused mean reduction.
  • Construct the Yoon Kim TextCNN architecture with parallel multi-scale 1D convolutions.
  • Implement 1-max pooling over time to achieve variable-length sequence invariance.
  • Verify dimensional compatibility and classification output shapes.

Prerequisites

  • Day 222 (Sequence-to-Sequence and Early Attention).
  • Python 3.11+ with PyTorch and NumPy.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 1 GB RAM.
  • 100 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

PyTorch is open-source software maintained by the Linux Foundation under a modified BSD license.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/text_classification_with_embeddings_lib.py: Student scaffold file.
  • examples/text_classification_with_embeddings_lib.py: Complete reference implementation.
  • tests/test_text_classification_with_embeddings_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/text_classification_with_embeddings_lib.py

What the commands do

  • Executes forward passes through EmbeddingBag and TextCNN models.
  • Verifies output tensor dimensions and multi-scale feature pooling.
  • Runs unit test assertions.

Expected output

Classification Demo: EB Shape = torch.Size([2, 2]), CNN Shape = torch.Size([2, 2])

Validation steps

  1. Verify EmbeddingBagClassifier.forward outputs a tensor of shape (batch_size, num_classes).
  2. Confirm TextCNN.forward outputs a tensor of shape (batch_size, num_classes).
  3. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Input sequence too short for TextCNN: Ensure sentence length is at least as large as the largest convolution kernel size ($k=3$ or $k=4$).

Security notes

All classification models execute in local memory.

Extension exercises

  1. Implement a multi-channel TextCNN with static and fine-tuned embeddings.
  2. Implement depthwise separable 1D convolutions to reduce parameters.

Expected output

FIELDS.md

# Expected Output Fields: Day 223

- `EB Shape`: Tensor shape of the EmbeddingBag classification logits.
- `CNN Shape`: Tensor shape of the TextCNN classification logits.

examples-run.txt

Classification Demo: EB Shape = torch.Size([2, 2]), CNN Shape = torch.Size([2, 2])

measured-values.txt

EB Shape: torch.Size([2, 2])
CNN Shape: torch.Size([2, 2])

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 2 items

tests/test_text_classification_with_embeddings_lib.py::test_embedding_bag_classifier PASSED [ 50%]
tests/test_text_classification_with_embeddings_lib.py::test_textcnn_classifier PASSED       [100%]

============================== 2 passed in 0.38s ===============================

Source files

conftest.py (585 bytes)
"""Make this lab importable when the suite runs.

pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""

import os
import sys

LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
    if os.path.isdir(candidate) and candidate not in sys.path:
        sys.path.insert(0, candidate)
examples/test_text_classification_with_embeddings_lib.py (615 bytes)
import pytest
import torch
from examples.text_classification_with_embeddings_lib import EmbeddingBagClassifier, TextCNN

def test_embedding_bag_classifier():
    torch.manual_seed(42)
    model = EmbeddingBagClassifier(vocab_size=100, embed_dim=16, num_classes=3)
    x = torch.randint(0, 100, (4, 10))
    logits = model(x)
    assert logits.shape == (4, 3)

def test_textcnn_classifier():
    torch.manual_seed(42)
    model = TextCNN(vocab_size=100, embed_dim=16, num_classes=2, num_filters=8, filter_sizes=[2, 3, 4])
    x = torch.randint(0, 100, (3, 8))
    logits = model(x)
    assert logits.shape == (3, 2)
examples/text_classification_with_embeddings_lib.py (2292 bytes)
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import List

class EmbeddingBagClassifier(nn.Module):
    def __init__(self, vocab_size: int, embed_dim: int, num_classes: int):
        super().__init__()
        self.embedding_bag = nn.EmbeddingBag(vocab_size, embed_dim, mode="mean")
        self.fc = nn.Linear(embed_dim, num_classes)

    def forward(self, text_indices: torch.Tensor, offsets: torch.Tensor = None) -> torch.Tensor:
        if offsets is None:
            # 2D tensor (Batch, Seq_Len)
            doc_embeds = self.embedding_bag(text_indices)
        else:
            doc_embeds = self.embedding_bag(text_indices, offsets)
        return self.fc(doc_embeds)

class TextCNN(nn.Module):
    def __init__(self, vocab_size: int, embed_dim: int, num_classes: int,
                 num_filters: int = 16, filter_sizes: List[int] = [2, 3]):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
        self.convs = nn.ModuleList([
            nn.Conv1d(in_channels=embed_dim, out_channels=num_filters, kernel_size=k)
            for k in filter_sizes
        ])
        self.fc = nn.Linear(num_filters * len(filter_sizes), num_classes)

    def forward(self, text_indices: torch.Tensor) -> torch.Tensor:
        # text_indices: (Batch, Seq_Len)
        embeds = self.embedding(text_indices).permute(0, 2, 1) # (Batch, embed_dim, Seq_Len)
        pooled_outputs = []
        for conv in self.convs:
            c = F.relu(conv(embeds))
            p = F.max_pool1d(c, kernel_size=c.size(2)).squeeze(2)
            pooled_outputs.append(p)
        
        doc_features = torch.cat(pooled_outputs, dim=1)
        return self.fc(doc_features)

def run_classification_demo():
    torch.manual_seed(42)
    eb_model = EmbeddingBagClassifier(vocab_size=50, embed_dim=8, num_classes=2)
    cnn_model = TextCNN(vocab_size=50, embed_dim=8, num_classes=2, num_filters=4, filter_sizes=[2, 3])

    inputs = torch.randint(1, 50, (2, 6)) # Batch=2, Len=6
    eb_logits = eb_model(inputs)
    cnn_logits = cnn_model(inputs)

    print(f"Classification Demo: EB Shape = {eb_logits.shape}, CNN Shape = {cnn_logits.shape}")
    return eb_logits, cnn_logits

if __name__ == "__main__":
    run_classification_demo()
metadata.yml (448 bytes)
lesson_id: D223
day: 223
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/text_classification_with_embeddings_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/test_text_classification_with_embeddings_lib.py (615 bytes)
import pytest
import torch
from examples.text_classification_with_embeddings_lib import EmbeddingBagClassifier, TextCNN

def test_embedding_bag_classifier():
    torch.manual_seed(42)
    model = EmbeddingBagClassifier(vocab_size=100, embed_dim=16, num_classes=3)
    x = torch.randint(0, 100, (4, 10))
    logits = model(x)
    assert logits.shape == (4, 3)

def test_textcnn_classifier():
    torch.manual_seed(42)
    model = TextCNN(vocab_size=100, embed_dim=16, num_classes=2, num_filters=8, filter_sizes=[2, 3, 4])
    x = torch.randint(0, 100, (3, 8))
    logits = model(x)
    assert logits.shape == (3, 2)
starter/text_classification_with_embeddings_lib.py (949 bytes)
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import List

class EmbeddingBagClassifier(nn.Module):
    def __init__(self, vocab_size: int, embed_dim: int, num_classes: int):
        super().__init__()
        # TODO: Initialize nn.EmbeddingBag with mode="mean" and linear head
        pass

    def forward(self, text_indices: torch.Tensor, offsets: torch.Tensor = None) -> torch.Tensor:
        # TODO: Compute mean document representation and logits
        pass

class TextCNN(nn.Module):
    def __init__(self, vocab_size: int, embed_dim: int, num_classes: int,
                 num_filters: int = 16, filter_sizes: List[int] = [2, 3]):
        super().__init__()
        # TODO: Initialize embedding, parallel Conv1d layers, and linear head
        pass

    def forward(self, text_indices: torch.Tensor) -> torch.Tensor:
        # TODO: Apply convolutions, 1-max pooling, and classification head
        pass
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 223 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_text_classification_with_embeddings_lib.py (615 bytes)
import pytest
import torch
from examples.text_classification_with_embeddings_lib import EmbeddingBagClassifier, TextCNN

def test_embedding_bag_classifier():
    torch.manual_seed(42)
    model = EmbeddingBagClassifier(vocab_size=100, embed_dim=16, num_classes=3)
    x = torch.randint(0, 100, (4, 10))
    logits = model(x)
    assert logits.shape == (4, 3)

def test_textcnn_classifier():
    torch.manual_seed(42)
    model = TextCNN(vocab_size=100, embed_dim=16, num_classes=2, num_filters=8, filter_sizes=[2, 3, 4])
    x = torch.randint(0, 100, (3, 8))
    logits = model(x)
    assert logits.shape == (3, 2)

Troubleshooting

Troubleshooting: Day 223 - Text Classification with Embeddings

Common Issues

  1. Dimension error in Conv1d:
    • Cause: Passing (Batch, Seq_Len, Dim) instead of (Batch, Dim, Seq_Len).
    • Fix: Use .permute(0, 2, 1).

Security notes

Security & Privacy: Day 223 - Text Classification with Embeddings

Security Guidance

  • All document classification executes in local memory.