Deployment, MLOps, and Security › Containers and Cloud › Day 331
Hands-on lab — Day 331: Dockerizing an AI Application
- ← Back to the Day 331 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/deployment-mlops-and-security/day-331-dockerizing-an-ai-application/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/image_demo.py Test
bash tests/run_tests.sh File tree
examples/image_demo.py examples/image_plan.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/image_plan.py tests/run_tests.sh tests/test_image_plan.py troubleshooting.md
Lab README
Day 331 Lab: Dockerizing an AI Application
Lesson
- Lesson title: Dockerizing an AI Application
- Day number: 331 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-331-dockerizing-an-ai-application
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-331-dockerizing-an-ai-applicationwhen the site is running.
Purpose
Work out how big an AI image will be, and what is making it big — before pulling twenty gigabytes to find out.
Packaging an AI application is not packaging a web application with a longer requirements file. Three things dominate: build tooling that is never needed at run time, a framework that is gigabytes before your code exists, and model weights that are frequently larger than everything else combined.
You will discover that the famous optimisation is the smaller one.
Learning objectives
- Compute an image's size from stages, and explain why only the final stage ships.
- Separate build-only components from runtime ones, which is what makes multi-stage work.
- Separate runtime components from data, which is what makes the larger saving possible.
- Convert image size into the unit that is felt: rollout time across a fleet.
- Recognise that most Docker guidance was written for applications without weights.
Prerequisites
- Day 330, "Docker Fundamentals" — layers, stages and the build cache.
- Comfortable with Python dataclasses and enums.
Supported operating systems
macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.
Hardware requirements
Any machine running Python 3.10 or newer. No GPU, no container runtime, no model download. The suite completes in hundredths of a second.
Required software
- Python 3.10 or newer.
pytest7 or newer.
Docker is not required and no weights are downloaded. The 13.5 GB in the demo is a number in a table — which is the point, since you should not have to move that much data to decide whether you should.
Free and open-source options
Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. For the real thing, Podman and Buildah build the same OCI images without a daemon, and docker buildx supports build secrets and cache mounts that keep a package cache out of the shipped layers.
Installation
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
Removing .venv fully reverses the installation.
File structure
starter/image_plan.py your work: ten stubbed tasks
examples/image_plan.py reference implementation
examples/image_demo.py the same service packaged three ways
tests/test_image_plan.py grouped by what the plan decides
tests/run_tests.sh suite entry point
expected-output/ real captured output and measured values
requirements/ pinned dependency
How to run
python3 examples/image_demo.py # size three packagings and compare
bash tests/run_tests.sh # run the suite
To work on the exercise, edit starter/image_plan.py, then copy it over the reference:
cp starter/image_plan.py examples/image_plan.py
bash tests/run_tests.sh
Keep a copy of the reference first if you want to compare.
What the commands do
python3 examples/image_demo.py sizes three packagings of one AI service — single stage, multi-stage, and multi-stage with the weights mounted — reports what is making each one big, and compares what each change bought in megabytes and in rollout seconds.
bash tests/run_tests.sh runs pytest over twenty tests grouped by what the plan decides: sizing, pull cost, review and comparison.
Expected output
--- what each change bought ---
multi-stage 21128 -> 18053 MB (3075 MB, 1.17x)
mount the weights 18053 -> 4553 MB (13500 MB, 3.97x)
both 21128 -> 4553 MB (16575 MB, 4.64x)
rollout to 20 nodes: 16902s -> 3642s
Read the second line against the first. The multi-stage build — the optimisation every Docker tutorial teaches — is the smaller win by a factor of four.
Validation steps
bash tests/run_tests.shreports20 passed.- Only the final stage may count toward the size. If the builder's four gigabytes appear in the total, multi-stage will appear to buy nothing.
cuda-develis BUILD andcuda-runtimeis RUNTIME. Swapping them produces either a broken image or a needlessly enormous one.- An unknown component must contribute
0, not raise. - A
final_stagenaming a stage that does not exist must raiseKeyErrorrather than silently using the first stage.
Tests
Twenty tests in four groups:
- sizing — a stage is its base plus its components; unknown components contribute nothing; only the final stage ships; carried components count; an unknown final stage raises.
- pull cost — time scales with size and with link speed; zero bandwidth does not divide by zero; a rollout pays once per node; zero or negative nodes cost nothing.
- review — build tooling in the final image is caught, including when carried; tooling confined to a builder is not; baked-in weights are reported with what they cost; a fat base is reported; a clean plan is clean.
- compare — savings and ratio; comparing a plan with itself saves nothing; moving the weights out beats the multi-stage build.
Cleanup
rm -rf __pycache__ .pytest_cache
rm -rf .venv
Both are safe to remove and fully reversible.
Troubleshooting
See troubleshooting.md.
Security notes
See security.md. In short: a smaller image is also a smaller attack surface, build tooling in a runtime image is a compiler in production, and baking a model you may not redistribute into an image you push is a licence question wearing engineering clothes.
Extension exercises
- Add layer-level granularity. Model each component as its own layer and compute what a rebuild transfers rather than what a first pull transfers. A registry that already holds most layers changes the picture considerably.
- Price it. Add an egress cost per gigabyte and compute what the 21 GB image costs per rollout across a fleet. Compare against the cost of the object storage the weights would have lived in.
- Size your own. Take a real AI service you have built, list its components with a purpose for each, and run it through the review. The interesting output is not the total — it is which single component you had never thought of as optional.
Navigation
- Lesson
- Previous: Day 330 — Docker Fundamentals
- Next: Day 332 — Docker Compose for Multi-Service Apps
Expected output
FIELDS.md
# Reading the image plan
## The plan line
image 21128 MB one pull 845.1s rollout to 20 nodes 16902.0s
| Part | Meaning |
| --- | --- |
| `image N MB` | The size of the **final** stage plus anything carried into it. Earlier stages are discarded. |
| `one pull` | How long a single node waits, at 200 Mbps. |
| `rollout to N nodes` | The pull paid once per node. This is where image size is actually felt. |
A 21 GB image is not "a bit slow to deploy". At twenty nodes it is four and a
half hours of pulling.
## The findings
| Rule | What it means | Typical saving |
| --- | --- | --- |
| `build-tooling-shipped` | Compilers and headers reached the final image. | 0.3–4 GB |
| `weights-baked-in` | Model weights are inside the image. | often the largest single item |
| `fat-base` | The final stage uses the full Python base rather than slim. | ~295 MB |
Each carries `saves_mb`, so the findings are ordered by what removing them
would actually buy rather than by how bad they sound.
## The comparison table
multi-stage 21128 -> 18053 MB (3075 MB, 1.17x)
mount the weights 18053 -> 4553 MB (13500 MB, 3.97x)
both 21128 -> 4553 MB (16575 MB, 4.64x)
Read the middle line against the first. **The multi-stage build — the
optimisation every Docker tutorial teaches — is the smaller of the two wins
here.** Moving the weights out of the image saves four times as much.
That is not an argument against multi-stage builds. It is an argument against
stopping there, which is what most guidance does, because most guidance was
written for web applications where there are no weights.
## The three component purposes
| Purpose | Meaning | May it be dropped from the final image? |
| --- | --- | --- |
| `BUILD` | Needed to produce artefacts, never at run time. | Yes — that is what a builder stage is for. |
| `RUNTIME` | The running process needs it. | No. |
| `DATA` | Weights and assets. Needed at run time. | Not dropped — **relocated** to a volume or object store. |
The distinction between `BUILD` and `RUNTIME` is what makes multi-stage
possible. The distinction between `RUNTIME` and `DATA` is what makes the bigger
saving possible, and it is the one people do not draw.
Note that `cuda-devel` is `BUILD` and `cuda-runtime` is `RUNTIME`. Confusing
them is the most common way to produce either a broken image or a needlessly
enormous one.
examples-run.txt
--- 1. single stage, full base, weights baked in ---
image 21128 MB one pull 845.1s rollout to 20 nodes 16902.0s
[build-tooling-shipped] saves 4380 MB
build-only components in the final image (build-essential, cuda-devel); produc
[weights-baked-in] saves 13500 MB
model weights are in the image (13500 MB); every node pulls them on every roll
[fat-base] saves 295 MB
the final stage uses the full Python base; the slim base carries the same inte
--- 2. multi-stage, slim runtime, weights baked in ---
image 18053 MB one pull 722.1s rollout to 20 nodes 14442.0s
[weights-baked-in] saves 13500 MB
model weights are in the image (13500 MB); every node pulls them on every roll
--- 3. multi-stage, slim runtime, weights mounted ---
image 4553 MB one pull 182.1s rollout to 20 nodes 3642.0s
no findings
--- what each change bought ---
multi-stage 21128 -> 18053 MB (3075 MB, 1.17x)
mount the weights 18053 -> 4553 MB (13500 MB, 3.97x)
both 21128 -> 4553 MB (16575 MB, 4.64x)
rollout to 20 nodes: 16902s -> 3642s
measured-values.txt
Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.
python3 examples/image_demo.py -> 20 lines, exit 0
bash tests/run_tests.sh -> 20 passed in 0.01s
the untouched starter -> 20 failed in 0.06s
The same AI service packaged three ways:
1. single stage, full base, weights baked in 21128 MB
2. multi-stage, slim runtime, weights baked in 18053 MB
3. multi-stage, slim runtime, weights mounted 4553 MB
What each change bought:
multi-stage 21128 -> 18053 MB 3075 MB saved 1.17x
mount the weights 18053 -> 4553 MB 13500 MB saved 3.97x
both 21128 -> 4553 MB 16575 MB saved 4.64x
This is the number this lab exists for, and it is not the one people expect.
The multi-stage build is the optimisation every Docker tutorial teaches, and
here it is the SMALLER win by a factor of four. Moving the weights out of the
image saves 13.5 GB; removing the build toolchain and the fat base saves 3.1.
The reason is simply that most Docker guidance was written for web
applications, where the largest thing in the image is the framework. In an AI
image the largest thing is usually the model, and no amount of multi-stage
discipline touches it.
Where the size is actually felt:
rollout to 20 nodes 16902s -> 3642s
Four hours and 42 minutes down to one hour and one minute, for the same
service. A rollout pays the pull once per node, which is why image size is a
deployment-time property rather than a disk-space one.
One implementation note. cuda-devel (4100 MB) is BUILD and cuda-runtime
(1600 MB) is RUNTIME. Treating them as interchangeable produces either a
broken image, if you drop the runtime, or a needlessly enormous one, if you
keep the devel package. The review function relies on that distinction and so
does the whole multi-stage argument.
A second note on the test suite. An earlier version of
test_moving_the_weights_out_beats_the_multi_stage_build asserted that mounting
saves more than 3x what multi-stage saves. That failed against a fixture whose
single-stage image carried cuda-devel, where the multi-stage saving is unusually
large -- 4675 MB against 13500. The claim that actually holds is the ORDERING,
not a fixed ratio, and the test now asserts that.
run-output.txt
--- 1. single stage, full base, weights baked in ---
image 21128 MB one pull 845.1s rollout to 20 nodes 16902.0s
[build-tooling-shipped] saves 4380 MB
build-only components in the final image (build-essential, cuda-devel); produc
[weights-baked-in] saves 13500 MB
model weights are in the image (13500 MB); every node pulls them on every roll
[fat-base] saves 295 MB
the final stage uses the full Python base; the slim base carries the same inte
--- 2. multi-stage, slim runtime, weights baked in ---
image 18053 MB one pull 722.1s rollout to 20 nodes 14442.0s
[weights-baked-in] saves 13500 MB
model weights are in the image (13500 MB); every node pulls them on every roll
--- 3. multi-stage, slim runtime, weights mounted ---
image 4553 MB one pull 182.1s rollout to 20 nodes 3642.0s
no findings
--- what each change bought ---
multi-stage 21128 -> 18053 MB (3075 MB, 1.17x)
mount the weights 18053 -> 4553 MB (13500 MB, 3.97x)
both 21128 -> 4553 MB (16575 MB, 4.64x)
rollout to 20 nodes: 16902s -> 3642s
starter-run.txt
An untouched starter fails the suite. Captured from a real run:
FAILED tests/test_image_plan.py::test_a_rollout_to_no_nodes_costs_nothing - N...
FAILED tests/test_image_plan.py::test_build_tooling_in_the_final_image_is_reported
FAILED tests/test_image_plan.py::test_build_tooling_confined_to_an_earlier_stage_is_not_reported
FAILED tests/test_image_plan.py::test_baked_in_weights_are_reported_with_what_they_cost
FAILED tests/test_image_plan.py::test_mounted_weights_produce_no_finding - No...
FAILED tests/test_image_plan.py::test_a_full_base_in_the_final_stage_is_reported
FAILED tests/test_image_plan.py::test_a_clean_plan_has_no_findings - NotImple...
FAILED tests/test_image_plan.py::test_carried_build_tooling_is_caught_too - N...
FAILED tests/test_image_plan.py::test_compare_reports_what_was_saved_and_the_ratio
FAILED tests/test_image_plan.py::test_comparing_a_plan_with_itself_saves_nothing
FAILED tests/test_image_plan.py::test_moving_the_weights_out_beats_the_multi_stage_build
20 failed in 0.06s
Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.
test-output.txt
.................... [100%]
20 passed in 0.01s
test-run.txt
.................... [100%]
20 passed in 0.01s
Source files
examples/image_demo.py (3259 bytes)
#!/usr/bin/env python3
"""The same AI service packaged three ways."""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from image_plan import (
Component,
ImagePlan,
Purpose,
Stage,
compare,
pull_seconds,
review,
rollout_seconds,
)
NODES = 20
RUNTIME = [
Component("torch", Purpose.RUNTIME),
Component("transformers", Purpose.RUNTIME),
Component("fastapi", Purpose.RUNTIME),
Component("app-code", Purpose.RUNTIME),
]
# 1. One stage, full base, build tools and weights all in the shipped image.
NAIVE = ImagePlan(
stages=[
Stage("final", "python-full", [
Component("build-essential", Purpose.BUILD),
Component("cuda-devel", Purpose.BUILD),
*RUNTIME,
Component("model-weights-7b", Purpose.DATA),
])
],
final_stage="final",
)
# 2. Multi-stage: compile in a builder, ship a slim runtime. Weights still in.
STAGED = ImagePlan(
stages=[
Stage("builder", "python-full", [
Component("build-essential", Purpose.BUILD),
Component("cuda-devel", Purpose.BUILD),
]),
Stage("final", "python-slim", [
Component("cuda-runtime", Purpose.RUNTIME),
*RUNTIME,
Component("model-weights-7b", Purpose.DATA),
]),
],
final_stage="final",
)
# 3. Same as 2, with the weights mounted at run time instead of baked in.
MOUNTED = ImagePlan(
stages=[
Stage("builder", "python-full", [
Component("build-essential", Purpose.BUILD),
Component("cuda-devel", Purpose.BUILD),
]),
Stage("final", "python-slim", [
Component("cuda-runtime", Purpose.RUNTIME),
*RUNTIME,
]),
],
final_stage="final",
)
def show(name: str, plan: ImagePlan) -> None:
print(f"--- {name} ---")
print(f" image {plan.size_mb:>6} MB one pull {pull_seconds(plan.size_mb):>7.1f}s"
f" rollout to {NODES} nodes {rollout_seconds(plan.size_mb, NODES):>8.1f}s")
found = review(plan)
for f in found:
print(f" [{f.rule}] saves {f.saves_mb} MB")
print(f" {f.message[:78]}")
if not found:
print(" no findings")
def main() -> int:
show("1. single stage, full base, weights baked in", NAIVE)
show("2. multi-stage, slim runtime, weights baked in", STAGED)
show("3. multi-stage, slim runtime, weights mounted", MOUNTED)
print("--- what each change bought ---")
a = compare(NAIVE, STAGED, nodes=NODES)
print(f" multi-stage {a['before_mb']:>6} -> {a['after_mb']:>6} MB "
f"({a['saved_mb']} MB, {a['ratio']}x)")
b = compare(STAGED, MOUNTED, nodes=NODES)
print(f" mount the weights{b['before_mb']:>6} -> {b['after_mb']:>6} MB "
f"({b['saved_mb']} MB, {b['ratio']}x)")
c = compare(NAIVE, MOUNTED, nodes=NODES)
print(f" both {c['before_mb']:>6} -> {c['after_mb']:>6} MB "
f"({c['saved_mb']} MB, {c['ratio']}x)")
print(f" rollout to {NODES} nodes: {c['before_rollout_s']:.0f}s -> {c['after_rollout_s']:.0f}s")
return 0
if __name__ == "__main__":
raise SystemExit(main())
examples/image_plan.py (5784 bytes)
"""Work out how big an AI image will be, and what is making it big.
Offline and standard-library only. Docker is not required: the input is a
described build -- stages, what each installs, and what the final stage keeps
-- so the arithmetic can be checked without pulling six gigabytes to find out.
Packaging an AI application is not packaging a web app with a bigger
requirements file. Three things dominate, and only one of them is obvious:
build tooling compilers and headers needed to BUILD wheels, never to run
the framework torch and CUDA runtime are gigabytes before your code exists
model weights frequently larger than everything else combined
The first is solved by a multi-stage build. The third is usually solved by NOT
solving it -- weights baked into an image are pulled onto every node on every
rollout, and a volume or object store is almost always the right answer.
Nothing here judges whether an image works. It computes what it costs to pull,
which is the number that decides how long a deploy takes.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
# Nominal compressed sizes in megabytes. The absolute values are illustrative;
# the ORDER OF MAGNITUDE between them is the part that drives every decision.
COMPONENT_MB = {
"python-slim": 45,
"python-full": 340,
"build-essential": 280, # gcc, make, headers -- build only
"cuda-runtime": 1_600,
"cuda-devel": 4_100, # build only
"torch": 2_400,
"transformers": 480,
"fastapi": 25,
"app-code": 3,
"model-weights-7b": 13_500,
"model-weights-small": 420,
}
class Purpose(str, Enum):
"""Why a component is present, which decides whether it may be dropped."""
BUILD = "build" # needed to produce artefacts, never at run time
RUNTIME = "runtime" # needed by the running process
DATA = "data" # weights and assets -- runtime, but relocatable
@dataclass(frozen=True)
class Component:
name: str
purpose: Purpose
@property
def size_mb(self) -> int:
return COMPONENT_MB.get(self.name, 0)
@dataclass
class Stage:
name: str
base: str
components: list[Component] = field(default_factory=list)
@property
def size_mb(self) -> int:
return COMPONENT_MB.get(self.base, 0) + sum(c.size_mb for c in self.components)
@dataclass
class ImagePlan:
"""A build described as stages plus what the final stage carries forward."""
stages: list[Stage]
final_stage: str
carried: list[Component] = field(default_factory=list)
@property
def final(self) -> Stage:
for s in self.stages:
if s.name == self.final_stage:
return s
raise KeyError(f"no stage named {self.final_stage!r}")
@property
def size_mb(self) -> int:
"""Only the final stage ships. Earlier stages are discarded."""
return self.final.size_mb + sum(c.size_mb for c in self.carried)
@property
def build_only_shipped(self) -> list[Component]:
"""Build tooling that reached the final image. Pure waste."""
present = self.final.components + self.carried
return [c for c in present if c.purpose is Purpose.BUILD]
@property
def weights_shipped(self) -> list[Component]:
present = self.final.components + self.carried
return [c for c in present if c.purpose is Purpose.DATA]
def pull_seconds(size_mb: int, mbps: float = 200.0) -> float:
"""How long one node waits to pull this image, at a given link speed."""
if mbps <= 0:
return 0.0
return round(size_mb * 8 / mbps, 1)
def rollout_seconds(size_mb: int, nodes: int, mbps: float = 200.0) -> float:
"""A rollout pulls onto every node. This is where image size is felt."""
return round(pull_seconds(size_mb, mbps) * max(0, nodes), 1)
@dataclass(frozen=True)
class Finding:
rule: str
message: str
saves_mb: int = 0
def review(plan: ImagePlan) -> list[Finding]:
"""What is making this image big, and what removing it would save."""
out: list[Finding] = []
build = plan.build_only_shipped
if build:
waste = sum(c.size_mb for c in build)
names = ", ".join(c.name for c in build)
out.append(
Finding(
"build-tooling-shipped",
f"build-only components in the final image ({names}); "
"produce them in an earlier stage and copy the artefacts out",
waste,
)
)
weights = plan.weights_shipped
if weights:
waste = sum(c.size_mb for c in weights)
out.append(
Finding(
"weights-baked-in",
f"model weights are in the image ({sum(c.size_mb for c in weights)} MB); "
"every node pulls them on every rollout even when only your code changed",
waste,
)
)
if plan.final.base == "python-full":
out.append(
Finding(
"fat-base",
"the final stage uses the full Python base; the slim base carries the same "
"interpreter without the build toolchain",
COMPONENT_MB["python-full"] - COMPONENT_MB["python-slim"],
)
)
return out
def compare(before: ImagePlan, after: ImagePlan, *, nodes: int = 20) -> dict[str, float]:
"""What the change bought, in the units that are actually felt."""
b, a = before.size_mb, after.size_mb
return {
"before_mb": b,
"after_mb": a,
"saved_mb": b - a,
"ratio": round(b / a, 2) if a else 0.0,
"before_rollout_s": rollout_seconds(b, nodes),
"after_rollout_s": rollout_seconds(a, nodes),
}
metadata.yml (929 bytes)
lesson_id: D331
day: 331
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python3 examples/image_demo.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3 examples/image_demo.py
reproduced the documented figures exactly (21128 MB -> 18053 MB -> 4553 MB; multi-stage 1.17x,
moving the weights out 3.97x, both together 4.64x; a 20-node rollout falling from 16902s to 3642s);
bash tests/run_tests.sh -> 20 passed in 0.01s. The untouched starter fails all 20. requires_network
is false and Docker is NOT required: the input is a described build, so the arithmetic is checkable
without pulling twenty gigabytes to find out.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/image_plan.py (4372 bytes)
"""STARTER -- implement each TASK below, then run the tests.
Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
Work out how big an AI image will be, and what is making it big.
Offline and standard-library only. Docker is not required: the input is a
described build -- stages, what each installs, and what the final stage keeps
-- so the arithmetic can be checked without pulling six gigabytes to find out.
Packaging an AI application is not packaging a web app with a bigger
requirements file. Three things dominate, and only one of them is obvious:
build tooling compilers and headers needed to BUILD wheels, never to run
the framework torch and CUDA runtime are gigabytes before your code exists
model weights frequently larger than everything else combined
The first is solved by a multi-stage build. The third is usually solved by NOT
solving it -- weights baked into an image are pulled onto every node on every
rollout, and a volume or object store is almost always the right answer.
Nothing here judges whether an image works. It computes what it costs to pull,
which is the number that decides how long a deploy takes.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
COMPONENT_MB = {'python-slim': 45, 'python-full': 340, 'build-essential': 280, 'cuda-runtime': 1600, 'cuda-devel': 4100, 'torch': 2400, 'transformers': 480, 'fastapi': 25, 'app-code': 3, 'model-weights-7b': 13500, 'model-weights-small': 420}
class Purpose(str, Enum):
"""Why a component is present, which decides whether it may be dropped."""
BUILD = 'build'
RUNTIME = 'runtime'
DATA = 'data'
@dataclass(frozen=True)
class Component:
name: str
purpose: Purpose
@property
def size_mb(self) -> int:
raise NotImplementedError('TASK 1: implement size_mb.')
@dataclass
class Stage:
name: str
base: str
components: list[Component] = field(default_factory=list)
@property
def size_mb(self) -> int:
raise NotImplementedError('TASK 2: implement size_mb.')
@dataclass
class ImagePlan:
"""A build described as stages plus what the final stage carries forward."""
stages: list[Stage]
final_stage: str
carried: list[Component] = field(default_factory=list)
@property
def final(self) -> Stage:
raise NotImplementedError('TASK 3: implement final.')
@property
def size_mb(self) -> int:
"""Only the final stage ships. Earlier stages are discarded."""
raise NotImplementedError('TASK 4: implement size_mb. Only the final stage ships. Earlier stages are discarded.')
@property
def build_only_shipped(self) -> list[Component]:
"""Build tooling that reached the final image. Pure waste."""
raise NotImplementedError('TASK 5: implement build_only_shipped. Build tooling that reached the final image. Pure waste.')
@property
def weights_shipped(self) -> list[Component]:
raise NotImplementedError('TASK 6: implement weights_shipped.')
def pull_seconds(size_mb: int, mbps: float=200.0) -> float:
"""How long one node waits to pull this image, at a given link speed."""
raise NotImplementedError('TASK 7: implement pull_seconds. How long one node waits to pull this image, at a given link speed.')
def rollout_seconds(size_mb: int, nodes: int, mbps: float=200.0) -> float:
"""A rollout pulls onto every node. This is where image size is felt."""
raise NotImplementedError('TASK 8: implement rollout_seconds. A rollout pulls onto every node. This is where image size is felt.')
@dataclass(frozen=True)
class Finding:
rule: str
message: str
saves_mb: int = 0
def review(plan: ImagePlan) -> list[Finding]:
"""What is making this image big, and what removing it would save."""
raise NotImplementedError('TASK 9: implement review. What is making this image big, and what removing it would save.')
def compare(before: ImagePlan, after: ImagePlan, *, nodes: int=20) -> dict[str, float]:
"""What the change bought, in the units that are actually felt."""
raise NotImplementedError('TASK 10: implement compare. What the change bought, in the units that are actually felt.')
tests/run_tests.sh (200 bytes)
#!/usr/bin/env bash
# Run the day 331 image-plan test suite.
# Offline, standard library plus pytest. Docker is not required.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_image_plan.py (7025 bytes)
"""Grouped by what the image plan decides.
Run with: bash tests/run_tests.sh
"""
from __future__ import annotations
import os
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "examples"))
from image_plan import ( # noqa: E402
COMPONENT_MB,
Component,
ImagePlan,
Purpose,
Stage,
compare,
pull_seconds,
review,
rollout_seconds,
)
def rt(*names):
return [Component(n, Purpose.RUNTIME) for n in names]
def single(base="python-slim", comps=None):
return ImagePlan(stages=[Stage("final", base, comps or [])], final_stage="final")
# ------------------------------------------------------------------ sizing
def test_a_stage_is_its_base_plus_its_components():
s = Stage("final", "python-slim", rt("fastapi", "app-code"))
assert s.size_mb == COMPONENT_MB["python-slim"] + COMPONENT_MB["fastapi"] + COMPONENT_MB["app-code"]
def test_an_unknown_component_contributes_nothing_rather_than_raising():
assert Component("not-a-real-package", Purpose.RUNTIME).size_mb == 0
def test_only_the_final_stage_ships():
# THE point of a multi-stage build: the builder is discarded entirely.
plan = ImagePlan(
stages=[
Stage("builder", "python-full", [Component("cuda-devel", Purpose.BUILD)]),
Stage("final", "python-slim", rt("app-code")),
],
final_stage="final",
)
assert plan.size_mb == COMPONENT_MB["python-slim"] + COMPONENT_MB["app-code"]
assert COMPONENT_MB["cuda-devel"] not in (plan.size_mb,)
def test_carried_components_do_count_toward_the_final_size():
plan = ImagePlan(
stages=[Stage("final", "python-slim", [])],
final_stage="final",
carried=rt("torch"),
)
assert plan.size_mb == COMPONENT_MB["python-slim"] + COMPONENT_MB["torch"]
def test_naming_a_stage_that_does_not_exist_is_an_error():
plan = ImagePlan(stages=[Stage("final", "python-slim", [])], final_stage="typo")
try:
plan.size_mb
except KeyError:
return
raise AssertionError("expected a KeyError for an unknown final stage")
# -------------------------------------------------------------- pull cost
def test_pull_time_scales_with_size():
assert pull_seconds(200, mbps=200.0) == 8.0
assert pull_seconds(400, mbps=200.0) == 16.0
def test_a_faster_link_pulls_proportionally_faster():
assert pull_seconds(1000, mbps=100.0) == 2 * pull_seconds(1000, mbps=200.0)
def test_a_nonsensical_link_speed_does_not_divide_by_zero():
assert pull_seconds(1000, mbps=0.0) == 0.0
def test_a_rollout_pays_the_pull_once_per_node():
assert rollout_seconds(1000, 10) == 10 * pull_seconds(1000)
def test_a_rollout_to_no_nodes_costs_nothing():
assert rollout_seconds(1000, 0) == 0.0
assert rollout_seconds(1000, -5) == 0.0
# ------------------------------------------------------------------ review
def test_build_tooling_in_the_final_image_is_reported():
plan = single(comps=[Component("cuda-devel", Purpose.BUILD), *rt("app-code")])
f = next(x for x in review(plan) if x.rule == "build-tooling-shipped")
assert f.saves_mb == COMPONENT_MB["cuda-devel"]
def test_build_tooling_confined_to_an_earlier_stage_is_not_reported():
plan = ImagePlan(
stages=[
Stage("builder", "python-full", [Component("cuda-devel", Purpose.BUILD)]),
Stage("final", "python-slim", rt("app-code")),
],
final_stage="final",
)
assert "build-tooling-shipped" not in {f.rule for f in review(plan)}
def test_baked_in_weights_are_reported_with_what_they_cost():
plan = single(comps=[Component("model-weights-7b", Purpose.DATA), *rt("app-code")])
f = next(x for x in review(plan) if x.rule == "weights-baked-in")
assert f.saves_mb == COMPONENT_MB["model-weights-7b"]
def test_mounted_weights_produce_no_finding():
assert "weights-baked-in" not in {f.rule for f in review(single(comps=rt("app-code")))}
def test_a_full_base_in_the_final_stage_is_reported():
f = next(x for x in review(single(base="python-full", comps=rt("app-code"))) if x.rule == "fat-base")
assert f.saves_mb == COMPONENT_MB["python-full"] - COMPONENT_MB["python-slim"]
def test_a_clean_plan_has_no_findings():
assert review(single(comps=rt("cuda-runtime", "torch", "fastapi", "app-code"))) == []
def test_carried_build_tooling_is_caught_too():
# Copying a build artefact forward is fine; copying the toolchain is not.
plan = ImagePlan(
stages=[Stage("final", "python-slim", rt("app-code"))],
final_stage="final",
carried=[Component("build-essential", Purpose.BUILD)],
)
assert "build-tooling-shipped" in {f.rule for f in review(plan)}
# ----------------------------------------------------------------- compare
def test_compare_reports_what_was_saved_and_the_ratio():
before = single(comps=[Component("model-weights-7b", Purpose.DATA), *rt("app-code")])
after = single(comps=rt("app-code"))
c = compare(before, after)
assert c["saved_mb"] == COMPONENT_MB["model-weights-7b"]
assert c["ratio"] > 1.0
def test_comparing_a_plan_with_itself_saves_nothing():
p = single(comps=rt("app-code"))
c = compare(p, p)
assert c["saved_mb"] == 0 and c["ratio"] == 1.0
def test_moving_the_weights_out_beats_the_multi_stage_build():
# The finding this lab exists for: multi-stage is the famous optimisation
# and it is the SMALLER one for an AI image.
one_stage = single(
base="python-full",
comps=[
Component("build-essential", Purpose.BUILD),
Component("cuda-devel", Purpose.BUILD),
*rt("cuda-runtime", "torch", "app-code"),
Component("model-weights-7b", Purpose.DATA),
],
)
staged = ImagePlan(
stages=[
Stage("builder", "python-full", [
Component("build-essential", Purpose.BUILD),
Component("cuda-devel", Purpose.BUILD),
]),
Stage("final", "python-slim", [
*rt("cuda-runtime", "torch", "app-code"),
Component("model-weights-7b", Purpose.DATA),
]),
],
final_stage="final",
)
mounted = ImagePlan(
stages=staged.stages[:1] + [Stage("final", "python-slim", rt("cuda-runtime", "torch", "app-code"))],
final_stage="final",
)
from_multistage = compare(one_stage, staged)["saved_mb"]
from_mounting = compare(staged, mounted)["saved_mb"]
# Moving the weights out is the bigger single win. How much bigger depends
# on how much build tooling the one-stage image carried, so the claim
# tested here is the ordering, not a fixed ratio -- an earlier version of
# this test asserted 3x and failed on a fixture that shipped cuda-devel,
# where the multi-stage saving is unusually large.
assert from_mounting > from_multistage
assert from_mounting == COMPONENT_MB["model-weights-7b"]
Troubleshooting
Troubleshooting — Day 331
NotImplementedError on every test
Expected. The starter stubs ten functions — see expected-output/starter-run.txt.
Start with Stage.size_mb; nothing else can be checked until a stage has a size.
test_only_the_final_stage_ships fails
The heart of a multi-stage build: earlier stages are discarded entirely. ImagePlan.size_mb must consult only the final stage plus whatever was carried into it — if you are summing every stage, the builder's four gigabytes are still in your total and multi-stage appears to buy nothing.
test_naming_a_stage_that_does_not_exist_is_an_error fails
Returning None or falling back to the first stage hides a typo in final_stage and produces a confidently wrong number. Raise KeyError.
An unknown component crashes instead of contributing zero
COMPONENT_MB.get(name, 0), not COMPONENT_MB[name]. A plan naming a package the table does not know about is a normal thing to write, and it should not stop the arithmetic.
test_a_nonsensical_link_speed_does_not_divide_by_zero fails
pull_seconds(size, mbps=0) must return 0.0 rather than raising. Zero bandwidth is nonsense input, and a tool that crashes on it is harder to use than one that returns a harmless value.
test_a_rollout_to_no_nodes_costs_nothing fails
Guard the node count with max(0, nodes). A negative count should not produce negative seconds.
build-tooling-shipped misses a carried component
review must look at the final stage's components and the carried list. Copying a build artefact forward is the point of a multi-stage build; copying the toolchain forward defeats it, and that is exactly the case worth catching.
test_a_clean_plan_has_no_findings fails
Something fires on a good plan. The usual cause is treating cuda-runtime as build tooling — it is RUNTIME, and the runtime is what a GPU service needs. Only cuda-devel is build-only.
The multi-stage saving looks bigger than the weights saving
That can be legitimate, and it is worth understanding rather than "fixing". If the single-stage image carried cuda-devel (about 4 GB) then removing it is a large win. The lab's demo does not, which is why mounting the weights dominates there. The test asserts the ordering that holds for the demo's shapes, not a fixed ratio.
Security notes
Security notes — Day 331
What this lab does and does not touch
- No Docker required and none is invoked. The input is a described build. Nothing is pulled, built or run.
- No network, no credentials, no registry login.
- No writes outside this directory. Only
__pycache__/,.pytest_cache/and, if you create it,.venv/. - No
sudo, no global installs. - No model weights are downloaded. The 13.5 GB in the demo is a number in a table, which is the point — you should not need to move that much data to reason about whether you should.
A smaller image is a smaller attack surface
The size argument and the security argument point the same way, which is unusual and worth using.
Build tooling in a runtime image is a compiler in production. cuda-devel, gcc, make and the headers exist to produce artefacts. Left in the final image they give anyone who achieves code execution a full toolchain to build with. A multi-stage build removes them for a reason that has nothing to do with megabytes.
A slim base has fewer packages to have vulnerabilities. Most CVEs reported against an application image are in components the application never calls. The cheapest way to stop being affected is not to ship them.
Every layer is kept — including the one with the weights. If a model is licensed such that you may not redistribute it, baking it into an image you push to a registry is redistribution, and pushing to a public registry is publication. This is a licence question that looks like an engineering decision.
Weights deserve their own answer
The demo's third plan mounts the weights rather than baking them in. Three consequences beyond size:
- The mount is now a supply-chain input. Whatever writes to that volume can replace your model. Day 347 covers this properly; the short version is that a checksum on load is cheap and worth it.
- A public image no longer contains a private model. Separating them means an image leak is not a model leak.
- Rotating the model no longer requires rebuilding the image, which removes the temptation to rebuild in a hurry without re-running the checks.
Run as a non-root user, and mean it
Day 330's linter checks for USER. It matters more here: an AI service typically has a writable cache directory, a mounted model volume and a network listener. Root inside the container plus a writable mount is the combination that turns a container escape from theoretical into practical.
Reversing everything this lab did
rm -rf __pycache__ .pytest_cache .venv
Nothing else was created, and nothing outside this directory was modified.