Deep Learning › Sequences and Text › Day 223
Hands-on lab — Day 223: Text Classification with Embeddings
- ← Back to the Day 223 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/deep-learning/day-223-text-classification-with-embeddings/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/text_classification_with_embeddings_lib.py Test
./tests/run_tests.sh File tree
conftest.py examples/test_text_classification_with_embeddings_lib.py examples/text_classification_with_embeddings_lib.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/starter-run.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/test_text_classification_with_embeddings_lib.py starter/text_classification_with_embeddings_lib.py tests/run_tests.sh tests/test_text_classification_with_embeddings_lib.py troubleshooting.md
Lab README
Lab: Day 223 -- Text Classification with Embeddings
Lesson
Day number: 223 of 365. Course: Course05-SS02 (Deep Learning - Sequence Models and Transformers). Topic: Neural Text Classification Architectures.
Purpose
Build and test modular neural text classification architectures in PyTorch. Implement the fast EmbeddingBagClassifier with fused average pooling, build the Yoon Kim TextCNN multi-scale 1D convolution model with max-over-time pooling, and verify classification logits across variable-length text batches.
Learning objectives
- Implement
nn.EmbeddingBagdocument representations with fused mean reduction. - Construct the Yoon Kim TextCNN architecture with parallel multi-scale 1D convolutions.
- Implement 1-max pooling over time to achieve variable-length sequence invariance.
- Verify dimensional compatibility and classification output shapes.
Prerequisites
- Day 222 (Sequence-to-Sequence and Early Attention).
- Python 3.11+ with PyTorch and NumPy.
Supported operating systems
- macOS (Apple Silicon / Intel)
- Linux (Ubuntu, Debian, Fedora, Arch)
- Windows 11 / WSL2
Hardware requirements
- 1+ CPU cores.
- 1 GB RAM.
- 100 MB disk space.
Required software
- Python 3.11 or newer.
- pip package manager.
- virtualenv or venv module.
Free and open-source options
PyTorch is open-source software maintained by the Linux Foundation under a modified BSD license.
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/text_classification_with_embeddings_lib.py: Student scaffold file.examples/text_classification_with_embeddings_lib.py: Complete reference implementation.tests/test_text_classification_with_embeddings_lib.py: Pytest automated validation suite.expected-output/: Verified output logs and baseline values.
How to run
Execute the reference demonstration script:
python3 examples/text_classification_with_embeddings_lib.py
What the commands do
- Executes forward passes through EmbeddingBag and TextCNN models.
- Verifies output tensor dimensions and multi-scale feature pooling.
- Runs unit test assertions.
Expected output
Classification Demo: EB Shape = torch.Size([2, 2]), CNN Shape = torch.Size([2, 2])
Validation steps
- Verify
EmbeddingBagClassifier.forwardoutputs a tensor of shape(batch_size, num_classes). - Confirm
TextCNN.forwardoutputs a tensor of shape(batch_size, num_classes). - Ensure all unit test assertions pass.
Tests
Run the test runner script:
./tests/run_tests.sh
Cleanup
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +
Troubleshooting
- Input sequence too short for TextCNN: Ensure sentence length is at least as large as the largest convolution kernel size ($k=3$ or $k=4$).
Security notes
All classification models execute in local memory.
Extension exercises
- Implement a multi-channel TextCNN with static and fine-tuned embeddings.
- Implement depthwise separable 1D convolutions to reduce parameters.
Navigation
- Lesson title: Text Classification with Embeddings
- Day number: 223 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-223-text-classification-with-embeddings
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-223-text-classification-with-embeddingswhen the site is running.
Expected output
FIELDS.md
# Expected Output Fields: Day 223
- `EB Shape`: Tensor shape of the EmbeddingBag classification logits.
- `CNN Shape`: Tensor shape of the TextCNN classification logits.
examples-run.txt
Classification Demo: EB Shape = torch.Size([2, 2]), CNN Shape = torch.Size([2, 2])
measured-values.txt
EB Shape: torch.Size([2, 2])
CNN Shape: torch.Size([2, 2])
starter-run.txt
Starter scaffold executed. Ready for student implementation.
test-run.txt
============================= test session starts ==============================
collected 2 items
tests/test_text_classification_with_embeddings_lib.py::test_embedding_bag_classifier PASSED [ 50%]
tests/test_text_classification_with_embeddings_lib.py::test_textcnn_classifier PASSED [100%]
============================== 2 passed in 0.38s ===============================
Source files
conftest.py (585 bytes)
"""Make this lab importable when the suite runs.
pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""
import os
import sys
LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
if os.path.isdir(candidate) and candidate not in sys.path:
sys.path.insert(0, candidate)
examples/test_text_classification_with_embeddings_lib.py (615 bytes)
import pytest
import torch
from examples.text_classification_with_embeddings_lib import EmbeddingBagClassifier, TextCNN
def test_embedding_bag_classifier():
torch.manual_seed(42)
model = EmbeddingBagClassifier(vocab_size=100, embed_dim=16, num_classes=3)
x = torch.randint(0, 100, (4, 10))
logits = model(x)
assert logits.shape == (4, 3)
def test_textcnn_classifier():
torch.manual_seed(42)
model = TextCNN(vocab_size=100, embed_dim=16, num_classes=2, num_filters=8, filter_sizes=[2, 3, 4])
x = torch.randint(0, 100, (3, 8))
logits = model(x)
assert logits.shape == (3, 2)
examples/text_classification_with_embeddings_lib.py (2292 bytes)
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import List
class EmbeddingBagClassifier(nn.Module):
def __init__(self, vocab_size: int, embed_dim: int, num_classes: int):
super().__init__()
self.embedding_bag = nn.EmbeddingBag(vocab_size, embed_dim, mode="mean")
self.fc = nn.Linear(embed_dim, num_classes)
def forward(self, text_indices: torch.Tensor, offsets: torch.Tensor = None) -> torch.Tensor:
if offsets is None:
# 2D tensor (Batch, Seq_Len)
doc_embeds = self.embedding_bag(text_indices)
else:
doc_embeds = self.embedding_bag(text_indices, offsets)
return self.fc(doc_embeds)
class TextCNN(nn.Module):
def __init__(self, vocab_size: int, embed_dim: int, num_classes: int,
num_filters: int = 16, filter_sizes: List[int] = [2, 3]):
super().__init__()
self.embedding = nn.Embedding(vocab_size, embed_dim, padding_idx=0)
self.convs = nn.ModuleList([
nn.Conv1d(in_channels=embed_dim, out_channels=num_filters, kernel_size=k)
for k in filter_sizes
])
self.fc = nn.Linear(num_filters * len(filter_sizes), num_classes)
def forward(self, text_indices: torch.Tensor) -> torch.Tensor:
# text_indices: (Batch, Seq_Len)
embeds = self.embedding(text_indices).permute(0, 2, 1) # (Batch, embed_dim, Seq_Len)
pooled_outputs = []
for conv in self.convs:
c = F.relu(conv(embeds))
p = F.max_pool1d(c, kernel_size=c.size(2)).squeeze(2)
pooled_outputs.append(p)
doc_features = torch.cat(pooled_outputs, dim=1)
return self.fc(doc_features)
def run_classification_demo():
torch.manual_seed(42)
eb_model = EmbeddingBagClassifier(vocab_size=50, embed_dim=8, num_classes=2)
cnn_model = TextCNN(vocab_size=50, embed_dim=8, num_classes=2, num_filters=4, filter_sizes=[2, 3])
inputs = torch.randint(1, 50, (2, 6)) # Batch=2, Len=6
eb_logits = eb_model(inputs)
cnn_logits = cnn_model(inputs)
print(f"Classification Demo: EB Shape = {eb_logits.shape}, CNN Shape = {cnn_logits.shape}")
return eb_logits, cnn_logits
if __name__ == "__main__":
run_classification_demo()
metadata.yml (448 bytes)
lesson_id: D223
day: 223
kind: lab
languages:
- python
setup_commands:
- 'pip install -r requirements/requirements.txt'
run_commands:
- 'python3 examples/text_classification_with_embeddings_lib.py'
test_commands:
- './tests/run_tests.sh'
cleanup_commands:
- 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/test_text_classification_with_embeddings_lib.py (615 bytes)
import pytest
import torch
from examples.text_classification_with_embeddings_lib import EmbeddingBagClassifier, TextCNN
def test_embedding_bag_classifier():
torch.manual_seed(42)
model = EmbeddingBagClassifier(vocab_size=100, embed_dim=16, num_classes=3)
x = torch.randint(0, 100, (4, 10))
logits = model(x)
assert logits.shape == (4, 3)
def test_textcnn_classifier():
torch.manual_seed(42)
model = TextCNN(vocab_size=100, embed_dim=16, num_classes=2, num_filters=8, filter_sizes=[2, 3, 4])
x = torch.randint(0, 100, (3, 8))
logits = model(x)
assert logits.shape == (3, 2)
starter/text_classification_with_embeddings_lib.py (949 bytes)
import torch
import torch.nn as nn
import torch.nn.functional as F
from typing import List
class EmbeddingBagClassifier(nn.Module):
def __init__(self, vocab_size: int, embed_dim: int, num_classes: int):
super().__init__()
# TODO: Initialize nn.EmbeddingBag with mode="mean" and linear head
pass
def forward(self, text_indices: torch.Tensor, offsets: torch.Tensor = None) -> torch.Tensor:
# TODO: Compute mean document representation and logits
pass
class TextCNN(nn.Module):
def __init__(self, vocab_size: int, embed_dim: int, num_classes: int,
num_filters: int = 16, filter_sizes: List[int] = [2, 3]):
super().__init__()
# TODO: Initialize embedding, parallel Conv1d layers, and linear head
pass
def forward(self, text_indices: torch.Tensor) -> torch.Tensor:
# TODO: Apply convolutions, 1-max pooling, and classification head
pass
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 223 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_text_classification_with_embeddings_lib.py (615 bytes)
import pytest
import torch
from examples.text_classification_with_embeddings_lib import EmbeddingBagClassifier, TextCNN
def test_embedding_bag_classifier():
torch.manual_seed(42)
model = EmbeddingBagClassifier(vocab_size=100, embed_dim=16, num_classes=3)
x = torch.randint(0, 100, (4, 10))
logits = model(x)
assert logits.shape == (4, 3)
def test_textcnn_classifier():
torch.manual_seed(42)
model = TextCNN(vocab_size=100, embed_dim=16, num_classes=2, num_filters=8, filter_sizes=[2, 3, 4])
x = torch.randint(0, 100, (3, 8))
logits = model(x)
assert logits.shape == (3, 2)
Troubleshooting
Troubleshooting: Day 223 - Text Classification with Embeddings
Common Issues
- Dimension error in Conv1d:
- Cause: Passing
(Batch, Seq_Len, Dim)instead of(Batch, Dim, Seq_Len). - Fix: Use
.permute(0, 2, 1).
- Cause: Passing
Security notes
Security & Privacy: Day 223 - Text Classification with Embeddings
Security Guidance
- All document classification executes in local memory.