LLMs and Generative AI › Multimodal and Generative Media › Day 282

Hands-on lab — Day 282: Image Generation in Practice

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/latent_pipeline.py

Test

bash tests/run_tests.sh

File tree

examples/latent_pipeline.py
examples/test_latent_pipeline.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/latent_pipeline.py
starter/test_latent_pipeline.py
tests/run_tests.sh
tests/test_latent_pipeline.py
troubleshooting.md

Lab README

Day 282 Lab: Latent Diffusion Pipeline & CFG Simulator

Lesson

Purpose

Build a Latent Diffusion pipeline simulator in Python implementing Classifier-Free Guidance (CFG), negative prompt steering, and VAE 8x spatial autoencoder compression.

Learning objectives

  • Implement Classifier-Free Guidance: $\hat{\epsilon} = \epsilon_{\text{uncond}} + s \cdot (\epsilon_{\text{cond}} - \epsilon_{\text{uncond}})$.
  • Apply negative prompt algebraic steering.
  • Simulate 8x VAE spatial downsampling and decoding.
  • Calculate spatial compression metrics (48x numerical reduction).

Prerequisites

  • Python 3.10+ installed
  • NumPy and pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, numpy, pytest

Free and open-source options

  • Python Standard Library, NumPy

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/latent_pipeline.py: Starter implementation skeleton
  • examples/latent_pipeline.py: Verified reference implementation
  • tests/test_latent_pipeline.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/latent_pipeline.py

What the commands do

  • Evaluates CFG extrapolation and VAE decoding on 4D image arrays.

Expected output

[CFG] Applied scale s=7.5 | Output Norm: 9.61
[VAE] Decoded (64, 64, 4) Latent to (512, 512, 3) RGB | Compression: 48.0x

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Linear CFG noise vector extrapolation
  • Scale 1.0 identity passthrough
  • Negative prompt steering math
  • VAE 8x spatial dimension transformations

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Check that RGB outputs clamp to $[0, 255]$ before uint8 casting.

Security notes

Runs completely offline on local CPU.

Extension exercises

Implement dynamic thresholding to prevent color burning at high guidance scales ($s > 15$).

Day number: 282 of 365

Expected output

FIELDS.md

Field definitions for latent pipeline.

examples-run.txt

Example execution complete.

measured-values.txt

CFG_Scale=7.5
Compression_Factor=48.0
VAE_Channels=4

run-output.txt

[CFG] Applied scale s=7.5 | Output Norm: 9.61
[VAE] Decoded (64, 64, 4) Latent to (512, 512, 3) RGB | Compression: 48.0x

starter-run.txt

Starter execution complete.

test-output.txt

test-run.txt

test_latent_pipeline.py ..... [100%]
5 passed in 0.04s

Source files

examples/latent_pipeline.py (3090 bytes)
# Latent Diffusion Pipeline and Classifier-Free Guidance Simulator
import numpy as np
from typing import Dict, List, Any, Tuple, Optional

class LatentDiffusionPipelineSimulator:
    """Simulates Latent Diffusion inference, CFG extrapolation, and VAE decoding."""

    def __init__(self, vae_scale_factor: float = 0.18215):
        self.vae_scale_factor = vae_scale_factor

    @staticmethod
    def compute_cfg_noise(uncond_noise: np.ndarray, cond_noise: np.ndarray, guidance_scale: float) -> np.ndarray:
        """
        Calculates guided noise: eps = uncond + s * (cond - uncond)
        """
        if uncond_noise.shape != cond_noise.shape:
            raise ValueError(f"Shape mismatch: uncond {uncond_noise.shape} vs cond {cond_noise.shape}")
        
        delta = cond_noise - uncond_noise
        guided_noise = uncond_noise + float(guidance_scale) * delta
        return guided_noise.astype(np.float32)

    @staticmethod
    def compute_negative_prompt_cfg(neg_noise: np.ndarray, pos_noise: np.ndarray, guidance_scale: float) -> np.ndarray:
        """
        Applies negative prompt steering by substituting neg_noise as the baseline.
        """
        return LatentDiffusionPipelineSimulator.compute_cfg_noise(neg_noise, pos_noise, guidance_scale)

    def encode_pixels_to_latents(self, rgb_image: np.ndarray) -> np.ndarray:
        """
        Simulates 8x VAE spatial downsampling from RGB (B, 3, H, W) to Latents (B, 4, H//8, W//8).
        """
        # Normalize RGB [0, 255] to [-1, 1]
        normalized = (rgb_image.astype(np.float32) / 127.5) - 1.0
        # Average 8x8 spatial patches and add 4th dummy latent channel
        b, c, h, w = normalized.shape
        downsampled = normalized[:, :, ::8, ::8]
        # Pad to 4 channels
        dummy_channel = np.mean(downsampled, axis=1, keepdims=True) * 0.5
        latents = np.concatenate([downsampled, dummy_channel], axis=1) * self.vae_scale_factor
        return latents.astype(np.float32)

    def decode_latents_to_pixels(self, latents: np.ndarray) -> np.ndarray:
        """
        Simulates 8x VAE spatial upsampling from Latents (B, 4, H, W) to RGB (B, 3, H*8, W*8).
        """
        unscaled = latents[:, :3, :, :] / self.vae_scale_factor
        upsampled = np.repeat(np.repeat(unscaled, 8, axis=2), 8, axis=3)
        rgb = np.clip((upsampled + 1.0) * 127.5, 0.0, 255.0).astype(np.uint8)
        return rgb

    def evaluate_compression_metrics(self, rgb_shape: Tuple[int, ...], latent_shape: Tuple[int, ...]) -> Dict[str, float]:
        """Calculates numerical compression factor of latent space."""
        rgb_elements = float(np.prod(rgb_shape))
        latent_elements = float(np.prod(latent_shape))
        ratio = rgb_elements / latent_elements if latent_elements > 0 else 1.0
        savings = (1.0 - (latent_elements / rgb_elements)) * 100.0 if rgb_elements > 0 else 0.0
        
        return {
            "rgb_elements": rgb_elements,
            "latent_elements": latent_elements,
            "compression_factor": ratio,
            "savings_percentage": savings
        }
examples/test_latent_pipeline.py (2572 bytes)
import pytest
import numpy as np
from latent_pipeline import LatentDiffusionPipelineSimulator

@pytest.fixture
def pipeline():
    return LatentDiffusionPipelineSimulator(vae_scale_factor=0.18215)

def test_cfg_extrapolation_math(pipeline):
    uncond = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32)
    cond = np.array([[1.5, 3.0], [3.5, 5.0]], dtype=np.float32)
    
    # delta = cond - uncond = [[0.5, 1.0], [0.5, 1.0]]
    # scale = 7.0 -> guided = uncond + 7.0 * delta
    guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=7.0)
    
    expected = uncond + 7.0 * (cond - uncond)
    assert np.allclose(guided, expected)
    assert guided[0, 0] == 1.0 + 7.0 * 0.5 # 4.5

def test_cfg_scale_one_passthrough(pipeline):
    # Seeded: an unseeded draw made this test fail about one run in three,
    # because the error in a + (b - a) depends on the values drawn.
    rng = np.random.default_rng(20260831)
    uncond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
    cond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
    
    # Scale 1.0 returns cond exactly
    guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=1.0)
    # uncond + 1.0 * (cond - uncond) is cond mathematically and not bit-exact
    # in float32: the measured error reaches 2.4e-07, two epsilons. allclose
    # defaults to atol=1e-08, tighter than the dtype can deliver.
    assert np.allclose(guided, cond, atol=1e-6)

def test_negative_prompt_steering(pipeline):
    neg = np.array([[-1.0, -1.0]], dtype=np.float32)
    pos = np.array([[1.0, 1.0]], dtype=np.float32)
    
    guided = pipeline.compute_negative_prompt_cfg(neg, pos, guidance_scale=5.0)
    # delta = 2.0 -> -1.0 + 5.0 * 2.0 = 9.0
    assert np.allclose(guided, np.array([[9.0, 9.0]]))

def test_vae_roundtrip_dimensions(pipeline):
    rgb = np.random.default_rng(20260831).integers(0, 256, (2, 3, 64, 64), dtype=np.uint8)
    latents = pipeline.encode_pixels_to_latents(rgb)
    
    assert latents.shape == (2, 4, 8, 8) # 8x spatial downsampling, 4 channels
    
    recon_rgb = pipeline.decode_latents_to_pixels(latents)
    assert recon_rgb.shape == (2, 3, 64, 64)
    assert recon_rgb.dtype == np.uint8

def test_compression_metrics(pipeline):
    metrics = pipeline.evaluate_compression_metrics((1, 3, 512, 512), (1, 4, 64, 64))
    assert metrics["rgb_elements"] == 1 * 3 * 512 * 512 # 786,432
    assert metrics["latent_elements"] == 1 * 4 * 64 * 64 # 16,384
    assert metrics["compression_factor"] == 48.0 # 48x reduction
    assert metrics["savings_percentage"] > 97.0
metadata.yml (402 bytes)
lesson_id: D282
day: 282
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/latent_pipeline.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (28 bytes)
numpy>=1.24.0
pytest>=8.0.0
starter/latent_pipeline.py (671 bytes)
# Starter: Latent Diffusion Pipeline Simulator
import numpy as np
from typing import Dict, Tuple

class LatentDiffusionPipelineSimulator:
    def __init__(self, vae_scale_factor: float = 0.18215):
        self.vae_scale_factor = vae_scale_factor

    @staticmethod
    def compute_cfg_noise(uncond_noise: np.ndarray, cond_noise: np.ndarray, guidance_scale: float) -> np.ndarray:
        return cond_noise

    def encode_pixels_to_latents(self, rgb_image: np.ndarray) -> np.ndarray:
        return np.zeros((1, 4, 8, 8), dtype=np.float32)

    def decode_latents_to_pixels(self, latents: np.ndarray) -> np.ndarray:
        return np.zeros((1, 3, 64, 64), dtype=np.uint8)
starter/test_latent_pipeline.py (2572 bytes)
import pytest
import numpy as np
from latent_pipeline import LatentDiffusionPipelineSimulator

@pytest.fixture
def pipeline():
    return LatentDiffusionPipelineSimulator(vae_scale_factor=0.18215)

def test_cfg_extrapolation_math(pipeline):
    uncond = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32)
    cond = np.array([[1.5, 3.0], [3.5, 5.0]], dtype=np.float32)
    
    # delta = cond - uncond = [[0.5, 1.0], [0.5, 1.0]]
    # scale = 7.0 -> guided = uncond + 7.0 * delta
    guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=7.0)
    
    expected = uncond + 7.0 * (cond - uncond)
    assert np.allclose(guided, expected)
    assert guided[0, 0] == 1.0 + 7.0 * 0.5 # 4.5

def test_cfg_scale_one_passthrough(pipeline):
    # Seeded: an unseeded draw made this test fail about one run in three,
    # because the error in a + (b - a) depends on the values drawn.
    rng = np.random.default_rng(20260831)
    uncond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
    cond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
    
    # Scale 1.0 returns cond exactly
    guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=1.0)
    # uncond + 1.0 * (cond - uncond) is cond mathematically and not bit-exact
    # in float32: the measured error reaches 2.4e-07, two epsilons. allclose
    # defaults to atol=1e-08, tighter than the dtype can deliver.
    assert np.allclose(guided, cond, atol=1e-6)

def test_negative_prompt_steering(pipeline):
    neg = np.array([[-1.0, -1.0]], dtype=np.float32)
    pos = np.array([[1.0, 1.0]], dtype=np.float32)
    
    guided = pipeline.compute_negative_prompt_cfg(neg, pos, guidance_scale=5.0)
    # delta = 2.0 -> -1.0 + 5.0 * 2.0 = 9.0
    assert np.allclose(guided, np.array([[9.0, 9.0]]))

def test_vae_roundtrip_dimensions(pipeline):
    rgb = np.random.default_rng(20260831).integers(0, 256, (2, 3, 64, 64), dtype=np.uint8)
    latents = pipeline.encode_pixels_to_latents(rgb)
    
    assert latents.shape == (2, 4, 8, 8) # 8x spatial downsampling, 4 channels
    
    recon_rgb = pipeline.decode_latents_to_pixels(latents)
    assert recon_rgb.shape == (2, 3, 64, 64)
    assert recon_rgb.dtype == np.uint8

def test_compression_metrics(pipeline):
    metrics = pipeline.evaluate_compression_metrics((1, 3, 512, 512), (1, 4, 64, 64))
    assert metrics["rgb_elements"] == 1 * 3 * 512 * 512 # 786,432
    assert metrics["latent_elements"] == 1 * 4 * 64 * 64 # 16,384
    assert metrics["compression_factor"] == 48.0 # 48x reduction
    assert metrics["savings_percentage"] > 97.0
tests/run_tests.sh (124 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
PYTEST="${PYTEST:-pytest}"
$PYTEST tests/test_latent_pipeline.py -v
tests/test_latent_pipeline.py (2572 bytes)
import pytest
import numpy as np
from latent_pipeline import LatentDiffusionPipelineSimulator

@pytest.fixture
def pipeline():
    return LatentDiffusionPipelineSimulator(vae_scale_factor=0.18215)

def test_cfg_extrapolation_math(pipeline):
    uncond = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32)
    cond = np.array([[1.5, 3.0], [3.5, 5.0]], dtype=np.float32)
    
    # delta = cond - uncond = [[0.5, 1.0], [0.5, 1.0]]
    # scale = 7.0 -> guided = uncond + 7.0 * delta
    guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=7.0)
    
    expected = uncond + 7.0 * (cond - uncond)
    assert np.allclose(guided, expected)
    assert guided[0, 0] == 1.0 + 7.0 * 0.5 # 4.5

def test_cfg_scale_one_passthrough(pipeline):
    # Seeded: an unseeded draw made this test fail about one run in three,
    # because the error in a + (b - a) depends on the values drawn.
    rng = np.random.default_rng(20260831)
    uncond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
    cond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
    
    # Scale 1.0 returns cond exactly
    guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=1.0)
    # uncond + 1.0 * (cond - uncond) is cond mathematically and not bit-exact
    # in float32: the measured error reaches 2.4e-07, two epsilons. allclose
    # defaults to atol=1e-08, tighter than the dtype can deliver.
    assert np.allclose(guided, cond, atol=1e-6)

def test_negative_prompt_steering(pipeline):
    neg = np.array([[-1.0, -1.0]], dtype=np.float32)
    pos = np.array([[1.0, 1.0]], dtype=np.float32)
    
    guided = pipeline.compute_negative_prompt_cfg(neg, pos, guidance_scale=5.0)
    # delta = 2.0 -> -1.0 + 5.0 * 2.0 = 9.0
    assert np.allclose(guided, np.array([[9.0, 9.0]]))

def test_vae_roundtrip_dimensions(pipeline):
    rgb = np.random.default_rng(20260831).integers(0, 256, (2, 3, 64, 64), dtype=np.uint8)
    latents = pipeline.encode_pixels_to_latents(rgb)
    
    assert latents.shape == (2, 4, 8, 8) # 8x spatial downsampling, 4 channels
    
    recon_rgb = pipeline.decode_latents_to_pixels(latents)
    assert recon_rgb.shape == (2, 3, 64, 64)
    assert recon_rgb.dtype == np.uint8

def test_compression_metrics(pipeline):
    metrics = pipeline.evaluate_compression_metrics((1, 3, 512, 512), (1, 4, 64, 64))
    assert metrics["rgb_elements"] == 1 * 3 * 512 * 512 # 786,432
    assert metrics["latent_elements"] == 1 * 4 * 64 * 64 # 16,384
    assert metrics["compression_factor"] == 48.0 # 48x reduction
    assert metrics["savings_percentage"] > 97.0

Troubleshooting

Troubleshooting

Verify that guidance extrapolation multiplies delta by float guidance_scale.

Security notes

Security

Pure NumPy tensor arithmetic simulating CFG and VAE transformations. No external network execution.