LLMs and Generative AI › Multimodal and Generative Media › Day 282
Hands-on lab — Day 282: Image Generation in Practice
- ← Back to the Day 282 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/llms-and-generative-ai/day-282-image-generation-in-practice/
Commands
Setup
pip install -r requirements/requirements.txt Run
python examples/latent_pipeline.py Test
bash tests/run_tests.sh File tree
examples/latent_pipeline.py examples/test_latent_pipeline.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/latent_pipeline.py starter/test_latent_pipeline.py tests/run_tests.sh tests/test_latent_pipeline.py troubleshooting.md
Lab README
Day 282 Lab: Latent Diffusion Pipeline & CFG Simulator
Lesson
- Lesson title: Image Generation in Practice
- Day number: 282 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-282-image-generation-in-practice
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-282-image-generation-in-practicewhen the site is running.
Purpose
Build a Latent Diffusion pipeline simulator in Python implementing Classifier-Free Guidance (CFG), negative prompt steering, and VAE 8x spatial autoencoder compression.
Learning objectives
- Implement Classifier-Free Guidance: $\hat{\epsilon} = \epsilon_{\text{uncond}} + s \cdot (\epsilon_{\text{cond}} - \epsilon_{\text{uncond}})$.
- Apply negative prompt algebraic steering.
- Simulate 8x VAE spatial downsampling and decoding.
- Calculate spatial compression metrics (48x numerical reduction).
Prerequisites
- Python 3.10+ installed
- NumPy and pytest installed
Supported operating systems
- macOS, Linux, Windows WSL2
Hardware requirements
- Standard CPU, 512MB RAM
Required software
- Python 3.10+, numpy, pytest
Free and open-source options
- Python Standard Library, NumPy
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/latent_pipeline.py: Starter implementation skeletonexamples/latent_pipeline.py: Verified reference implementationtests/test_latent_pipeline.py: Test suiteexpected-output/: Captured execution logs
How to run
python3 examples/latent_pipeline.py
What the commands do
- Evaluates CFG extrapolation and VAE decoding on 4D image arrays.
Expected output
[CFG] Applied scale s=7.5 | Output Norm: 9.61
[VAE] Decoded (64, 64, 4) Latent to (512, 512, 3) RGB | Compression: 48.0x
Validation steps
Run test runner:
bash tests/run_tests.sh
Tests
The test suite validates:
- Linear CFG noise vector extrapolation
- Scale 1.0 identity passthrough
- Negative prompt steering math
- VAE 8x spatial dimension transformations
Cleanup
rm -rf .pytest_cache __pycache__ .venv
Troubleshooting
Check that RGB outputs clamp to $[0, 255]$ before uint8 casting.
Security notes
Runs completely offline on local CPU.
Extension exercises
Implement dynamic thresholding to prevent color burning at high guidance scales ($s > 15$).
Navigation
Day number: 282 of 365
Expected output
FIELDS.md
Field definitions for latent pipeline. examples-run.txt
Example execution complete.
measured-values.txt
CFG_Scale=7.5
Compression_Factor=48.0
VAE_Channels=4
run-output.txt
[CFG] Applied scale s=7.5 | Output Norm: 9.61
[VAE] Decoded (64, 64, 4) Latent to (512, 512, 3) RGB | Compression: 48.0x
starter-run.txt
Starter execution complete.
test-output.txt
test-run.txt
test_latent_pipeline.py ..... [100%]
5 passed in 0.04s
Source files
examples/latent_pipeline.py (3090 bytes)
# Latent Diffusion Pipeline and Classifier-Free Guidance Simulator
import numpy as np
from typing import Dict, List, Any, Tuple, Optional
class LatentDiffusionPipelineSimulator:
"""Simulates Latent Diffusion inference, CFG extrapolation, and VAE decoding."""
def __init__(self, vae_scale_factor: float = 0.18215):
self.vae_scale_factor = vae_scale_factor
@staticmethod
def compute_cfg_noise(uncond_noise: np.ndarray, cond_noise: np.ndarray, guidance_scale: float) -> np.ndarray:
"""
Calculates guided noise: eps = uncond + s * (cond - uncond)
"""
if uncond_noise.shape != cond_noise.shape:
raise ValueError(f"Shape mismatch: uncond {uncond_noise.shape} vs cond {cond_noise.shape}")
delta = cond_noise - uncond_noise
guided_noise = uncond_noise + float(guidance_scale) * delta
return guided_noise.astype(np.float32)
@staticmethod
def compute_negative_prompt_cfg(neg_noise: np.ndarray, pos_noise: np.ndarray, guidance_scale: float) -> np.ndarray:
"""
Applies negative prompt steering by substituting neg_noise as the baseline.
"""
return LatentDiffusionPipelineSimulator.compute_cfg_noise(neg_noise, pos_noise, guidance_scale)
def encode_pixels_to_latents(self, rgb_image: np.ndarray) -> np.ndarray:
"""
Simulates 8x VAE spatial downsampling from RGB (B, 3, H, W) to Latents (B, 4, H//8, W//8).
"""
# Normalize RGB [0, 255] to [-1, 1]
normalized = (rgb_image.astype(np.float32) / 127.5) - 1.0
# Average 8x8 spatial patches and add 4th dummy latent channel
b, c, h, w = normalized.shape
downsampled = normalized[:, :, ::8, ::8]
# Pad to 4 channels
dummy_channel = np.mean(downsampled, axis=1, keepdims=True) * 0.5
latents = np.concatenate([downsampled, dummy_channel], axis=1) * self.vae_scale_factor
return latents.astype(np.float32)
def decode_latents_to_pixels(self, latents: np.ndarray) -> np.ndarray:
"""
Simulates 8x VAE spatial upsampling from Latents (B, 4, H, W) to RGB (B, 3, H*8, W*8).
"""
unscaled = latents[:, :3, :, :] / self.vae_scale_factor
upsampled = np.repeat(np.repeat(unscaled, 8, axis=2), 8, axis=3)
rgb = np.clip((upsampled + 1.0) * 127.5, 0.0, 255.0).astype(np.uint8)
return rgb
def evaluate_compression_metrics(self, rgb_shape: Tuple[int, ...], latent_shape: Tuple[int, ...]) -> Dict[str, float]:
"""Calculates numerical compression factor of latent space."""
rgb_elements = float(np.prod(rgb_shape))
latent_elements = float(np.prod(latent_shape))
ratio = rgb_elements / latent_elements if latent_elements > 0 else 1.0
savings = (1.0 - (latent_elements / rgb_elements)) * 100.0 if rgb_elements > 0 else 0.0
return {
"rgb_elements": rgb_elements,
"latent_elements": latent_elements,
"compression_factor": ratio,
"savings_percentage": savings
}
examples/test_latent_pipeline.py (2572 bytes)
import pytest
import numpy as np
from latent_pipeline import LatentDiffusionPipelineSimulator
@pytest.fixture
def pipeline():
return LatentDiffusionPipelineSimulator(vae_scale_factor=0.18215)
def test_cfg_extrapolation_math(pipeline):
uncond = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32)
cond = np.array([[1.5, 3.0], [3.5, 5.0]], dtype=np.float32)
# delta = cond - uncond = [[0.5, 1.0], [0.5, 1.0]]
# scale = 7.0 -> guided = uncond + 7.0 * delta
guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=7.0)
expected = uncond + 7.0 * (cond - uncond)
assert np.allclose(guided, expected)
assert guided[0, 0] == 1.0 + 7.0 * 0.5 # 4.5
def test_cfg_scale_one_passthrough(pipeline):
# Seeded: an unseeded draw made this test fail about one run in three,
# because the error in a + (b - a) depends on the values drawn.
rng = np.random.default_rng(20260831)
uncond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
cond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
# Scale 1.0 returns cond exactly
guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=1.0)
# uncond + 1.0 * (cond - uncond) is cond mathematically and not bit-exact
# in float32: the measured error reaches 2.4e-07, two epsilons. allclose
# defaults to atol=1e-08, tighter than the dtype can deliver.
assert np.allclose(guided, cond, atol=1e-6)
def test_negative_prompt_steering(pipeline):
neg = np.array([[-1.0, -1.0]], dtype=np.float32)
pos = np.array([[1.0, 1.0]], dtype=np.float32)
guided = pipeline.compute_negative_prompt_cfg(neg, pos, guidance_scale=5.0)
# delta = 2.0 -> -1.0 + 5.0 * 2.0 = 9.0
assert np.allclose(guided, np.array([[9.0, 9.0]]))
def test_vae_roundtrip_dimensions(pipeline):
rgb = np.random.default_rng(20260831).integers(0, 256, (2, 3, 64, 64), dtype=np.uint8)
latents = pipeline.encode_pixels_to_latents(rgb)
assert latents.shape == (2, 4, 8, 8) # 8x spatial downsampling, 4 channels
recon_rgb = pipeline.decode_latents_to_pixels(latents)
assert recon_rgb.shape == (2, 3, 64, 64)
assert recon_rgb.dtype == np.uint8
def test_compression_metrics(pipeline):
metrics = pipeline.evaluate_compression_metrics((1, 3, 512, 512), (1, 4, 64, 64))
assert metrics["rgb_elements"] == 1 * 3 * 512 * 512 # 786,432
assert metrics["latent_elements"] == 1 * 4 * 64 * 64 # 16,384
assert metrics["compression_factor"] == 48.0 # 48x reduction
assert metrics["savings_percentage"] > 97.0
metadata.yml (402 bytes)
lesson_id: D282
day: 282
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/latent_pipeline.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (28 bytes)
numpy>=1.24.0
pytest>=8.0.0
starter/latent_pipeline.py (671 bytes)
# Starter: Latent Diffusion Pipeline Simulator
import numpy as np
from typing import Dict, Tuple
class LatentDiffusionPipelineSimulator:
def __init__(self, vae_scale_factor: float = 0.18215):
self.vae_scale_factor = vae_scale_factor
@staticmethod
def compute_cfg_noise(uncond_noise: np.ndarray, cond_noise: np.ndarray, guidance_scale: float) -> np.ndarray:
return cond_noise
def encode_pixels_to_latents(self, rgb_image: np.ndarray) -> np.ndarray:
return np.zeros((1, 4, 8, 8), dtype=np.float32)
def decode_latents_to_pixels(self, latents: np.ndarray) -> np.ndarray:
return np.zeros((1, 3, 64, 64), dtype=np.uint8)
starter/test_latent_pipeline.py (2572 bytes)
import pytest
import numpy as np
from latent_pipeline import LatentDiffusionPipelineSimulator
@pytest.fixture
def pipeline():
return LatentDiffusionPipelineSimulator(vae_scale_factor=0.18215)
def test_cfg_extrapolation_math(pipeline):
uncond = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32)
cond = np.array([[1.5, 3.0], [3.5, 5.0]], dtype=np.float32)
# delta = cond - uncond = [[0.5, 1.0], [0.5, 1.0]]
# scale = 7.0 -> guided = uncond + 7.0 * delta
guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=7.0)
expected = uncond + 7.0 * (cond - uncond)
assert np.allclose(guided, expected)
assert guided[0, 0] == 1.0 + 7.0 * 0.5 # 4.5
def test_cfg_scale_one_passthrough(pipeline):
# Seeded: an unseeded draw made this test fail about one run in three,
# because the error in a + (b - a) depends on the values drawn.
rng = np.random.default_rng(20260831)
uncond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
cond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
# Scale 1.0 returns cond exactly
guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=1.0)
# uncond + 1.0 * (cond - uncond) is cond mathematically and not bit-exact
# in float32: the measured error reaches 2.4e-07, two epsilons. allclose
# defaults to atol=1e-08, tighter than the dtype can deliver.
assert np.allclose(guided, cond, atol=1e-6)
def test_negative_prompt_steering(pipeline):
neg = np.array([[-1.0, -1.0]], dtype=np.float32)
pos = np.array([[1.0, 1.0]], dtype=np.float32)
guided = pipeline.compute_negative_prompt_cfg(neg, pos, guidance_scale=5.0)
# delta = 2.0 -> -1.0 + 5.0 * 2.0 = 9.0
assert np.allclose(guided, np.array([[9.0, 9.0]]))
def test_vae_roundtrip_dimensions(pipeline):
rgb = np.random.default_rng(20260831).integers(0, 256, (2, 3, 64, 64), dtype=np.uint8)
latents = pipeline.encode_pixels_to_latents(rgb)
assert latents.shape == (2, 4, 8, 8) # 8x spatial downsampling, 4 channels
recon_rgb = pipeline.decode_latents_to_pixels(latents)
assert recon_rgb.shape == (2, 3, 64, 64)
assert recon_rgb.dtype == np.uint8
def test_compression_metrics(pipeline):
metrics = pipeline.evaluate_compression_metrics((1, 3, 512, 512), (1, 4, 64, 64))
assert metrics["rgb_elements"] == 1 * 3 * 512 * 512 # 786,432
assert metrics["latent_elements"] == 1 * 4 * 64 * 64 # 16,384
assert metrics["compression_factor"] == 48.0 # 48x reduction
assert metrics["savings_percentage"] > 97.0
tests/run_tests.sh (124 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
PYTEST="${PYTEST:-pytest}"
$PYTEST tests/test_latent_pipeline.py -v
tests/test_latent_pipeline.py (2572 bytes)
import pytest
import numpy as np
from latent_pipeline import LatentDiffusionPipelineSimulator
@pytest.fixture
def pipeline():
return LatentDiffusionPipelineSimulator(vae_scale_factor=0.18215)
def test_cfg_extrapolation_math(pipeline):
uncond = np.array([[1.0, 2.0], [3.0, 4.0]], dtype=np.float32)
cond = np.array([[1.5, 3.0], [3.5, 5.0]], dtype=np.float32)
# delta = cond - uncond = [[0.5, 1.0], [0.5, 1.0]]
# scale = 7.0 -> guided = uncond + 7.0 * delta
guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=7.0)
expected = uncond + 7.0 * (cond - uncond)
assert np.allclose(guided, expected)
assert guided[0, 0] == 1.0 + 7.0 * 0.5 # 4.5
def test_cfg_scale_one_passthrough(pipeline):
# Seeded: an unseeded draw made this test fail about one run in three,
# because the error in a + (b - a) depends on the values drawn.
rng = np.random.default_rng(20260831)
uncond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
cond = rng.standard_normal((2, 4, 8, 8)).astype(np.float32)
# Scale 1.0 returns cond exactly
guided = pipeline.compute_cfg_noise(uncond, cond, guidance_scale=1.0)
# uncond + 1.0 * (cond - uncond) is cond mathematically and not bit-exact
# in float32: the measured error reaches 2.4e-07, two epsilons. allclose
# defaults to atol=1e-08, tighter than the dtype can deliver.
assert np.allclose(guided, cond, atol=1e-6)
def test_negative_prompt_steering(pipeline):
neg = np.array([[-1.0, -1.0]], dtype=np.float32)
pos = np.array([[1.0, 1.0]], dtype=np.float32)
guided = pipeline.compute_negative_prompt_cfg(neg, pos, guidance_scale=5.0)
# delta = 2.0 -> -1.0 + 5.0 * 2.0 = 9.0
assert np.allclose(guided, np.array([[9.0, 9.0]]))
def test_vae_roundtrip_dimensions(pipeline):
rgb = np.random.default_rng(20260831).integers(0, 256, (2, 3, 64, 64), dtype=np.uint8)
latents = pipeline.encode_pixels_to_latents(rgb)
assert latents.shape == (2, 4, 8, 8) # 8x spatial downsampling, 4 channels
recon_rgb = pipeline.decode_latents_to_pixels(latents)
assert recon_rgb.shape == (2, 3, 64, 64)
assert recon_rgb.dtype == np.uint8
def test_compression_metrics(pipeline):
metrics = pipeline.evaluate_compression_metrics((1, 3, 512, 512), (1, 4, 64, 64))
assert metrics["rgb_elements"] == 1 * 3 * 512 * 512 # 786,432
assert metrics["latent_elements"] == 1 * 4 * 64 * 64 # 16,384
assert metrics["compression_factor"] == 48.0 # 48x reduction
assert metrics["savings_percentage"] > 97.0
Troubleshooting
Troubleshooting
Verify that guidance extrapolation multiplies delta by float guidance_scale.
Security notes
Security
Pure NumPy tensor arithmetic simulating CFG and VAE transformations. No external network execution.