Deployment, MLOps, and Security › Containers and Cloud › Day 331

Hands-on lab — Day 331: Dockerizing an AI Application

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/image_demo.py

Test

bash tests/run_tests.sh

File tree

examples/image_demo.py
examples/image_plan.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/image_plan.py
tests/run_tests.sh
tests/test_image_plan.py
troubleshooting.md

Lab README

Day 331 Lab: Dockerizing an AI Application

Lesson

Purpose

Work out how big an AI image will be, and what is making it big — before pulling twenty gigabytes to find out.

Packaging an AI application is not packaging a web application with a longer requirements file. Three things dominate: build tooling that is never needed at run time, a framework that is gigabytes before your code exists, and model weights that are frequently larger than everything else combined.

You will discover that the famous optimisation is the smaller one.

Learning objectives

  • Compute an image's size from stages, and explain why only the final stage ships.
  • Separate build-only components from runtime ones, which is what makes multi-stage work.
  • Separate runtime components from data, which is what makes the larger saving possible.
  • Convert image size into the unit that is felt: rollout time across a fleet.
  • Recognise that most Docker guidance was written for applications without weights.

Prerequisites

  • Day 330, "Docker Fundamentals" — layers, stages and the build cache.
  • Comfortable with Python dataclasses and enums.

Supported operating systems

macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.

Hardware requirements

Any machine running Python 3.10 or newer. No GPU, no container runtime, no model download. The suite completes in hundredths of a second.

Required software

  • Python 3.10 or newer.
  • pytest 7 or newer.

Docker is not required and no weights are downloaded. The 13.5 GB in the demo is a number in a table — which is the point, since you should not have to move that much data to decide whether you should.

Free and open-source options

Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. For the real thing, Podman and Buildah build the same OCI images without a daemon, and docker buildx supports build secrets and cache mounts that keep a package cache out of the shipped layers.

Installation

python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt

Removing .venv fully reverses the installation.

File structure

starter/image_plan.py     your work: ten stubbed tasks
examples/image_plan.py    reference implementation
examples/image_demo.py    the same service packaged three ways
tests/test_image_plan.py  grouped by what the plan decides
tests/run_tests.sh        suite entry point
expected-output/          real captured output and measured values
requirements/             pinned dependency

How to run

python3 examples/image_demo.py   # size three packagings and compare
bash tests/run_tests.sh          # run the suite

To work on the exercise, edit starter/image_plan.py, then copy it over the reference:

cp starter/image_plan.py examples/image_plan.py
bash tests/run_tests.sh

Keep a copy of the reference first if you want to compare.

What the commands do

python3 examples/image_demo.py sizes three packagings of one AI service — single stage, multi-stage, and multi-stage with the weights mounted — reports what is making each one big, and compares what each change bought in megabytes and in rollout seconds.

bash tests/run_tests.sh runs pytest over twenty tests grouped by what the plan decides: sizing, pull cost, review and comparison.

Expected output

--- what each change bought ---
  multi-stage       21128 ->  18053 MB  (3075 MB, 1.17x)
  mount the weights 18053 ->   4553 MB  (13500 MB, 3.97x)
  both              21128 ->   4553 MB  (16575 MB, 4.64x)
  rollout to 20 nodes: 16902s -> 3642s

Read the second line against the first. The multi-stage build — the optimisation every Docker tutorial teaches — is the smaller win by a factor of four.

Validation steps

  1. bash tests/run_tests.sh reports 20 passed.
  2. Only the final stage may count toward the size. If the builder's four gigabytes appear in the total, multi-stage will appear to buy nothing.
  3. cuda-devel is BUILD and cuda-runtime is RUNTIME. Swapping them produces either a broken image or a needlessly enormous one.
  4. An unknown component must contribute 0, not raise.
  5. A final_stage naming a stage that does not exist must raise KeyError rather than silently using the first stage.

Tests

Twenty tests in four groups:

  • sizing — a stage is its base plus its components; unknown components contribute nothing; only the final stage ships; carried components count; an unknown final stage raises.
  • pull cost — time scales with size and with link speed; zero bandwidth does not divide by zero; a rollout pays once per node; zero or negative nodes cost nothing.
  • review — build tooling in the final image is caught, including when carried; tooling confined to a builder is not; baked-in weights are reported with what they cost; a fat base is reported; a clean plan is clean.
  • compare — savings and ratio; comparing a plan with itself saves nothing; moving the weights out beats the multi-stage build.

Cleanup

rm -rf __pycache__ .pytest_cache
rm -rf .venv

Both are safe to remove and fully reversible.

Troubleshooting

See troubleshooting.md.

Security notes

See security.md. In short: a smaller image is also a smaller attack surface, build tooling in a runtime image is a compiler in production, and baking a model you may not redistribute into an image you push is a licence question wearing engineering clothes.

Extension exercises

  1. Add layer-level granularity. Model each component as its own layer and compute what a rebuild transfers rather than what a first pull transfers. A registry that already holds most layers changes the picture considerably.
  2. Price it. Add an egress cost per gigabyte and compute what the 21 GB image costs per rollout across a fleet. Compare against the cost of the object storage the weights would have lived in.
  3. Size your own. Take a real AI service you have built, list its components with a purpose for each, and run it through the review. The interesting output is not the total — it is which single component you had never thought of as optional.
  • Lesson
  • Previous: Day 330 — Docker Fundamentals
  • Next: Day 332 — Docker Compose for Multi-Service Apps

Expected output

FIELDS.md

# Reading the image plan

## The plan line

    image  21128 MB   one pull   845.1s   rollout to 20 nodes  16902.0s

| Part | Meaning |
| --- | --- |
| `image N MB` | The size of the **final** stage plus anything carried into it. Earlier stages are discarded. |
| `one pull` | How long a single node waits, at 200 Mbps. |
| `rollout to N nodes` | The pull paid once per node. This is where image size is actually felt. |

A 21 GB image is not "a bit slow to deploy". At twenty nodes it is four and a
half hours of pulling.

## The findings

| Rule | What it means | Typical saving |
| --- | --- | --- |
| `build-tooling-shipped` | Compilers and headers reached the final image. | 0.3–4 GB |
| `weights-baked-in` | Model weights are inside the image. | often the largest single item |
| `fat-base` | The final stage uses the full Python base rather than slim. | ~295 MB |

Each carries `saves_mb`, so the findings are ordered by what removing them
would actually buy rather than by how bad they sound.

## The comparison table

    multi-stage       21128 ->  18053 MB  (3075 MB, 1.17x)
    mount the weights 18053 ->   4553 MB  (13500 MB, 3.97x)
    both              21128 ->   4553 MB  (16575 MB, 4.64x)

Read the middle line against the first. **The multi-stage build — the
optimisation every Docker tutorial teaches — is the smaller of the two wins
here.** Moving the weights out of the image saves four times as much.

That is not an argument against multi-stage builds. It is an argument against
stopping there, which is what most guidance does, because most guidance was
written for web applications where there are no weights.

## The three component purposes

| Purpose | Meaning | May it be dropped from the final image? |
| --- | --- | --- |
| `BUILD` | Needed to produce artefacts, never at run time. | Yes — that is what a builder stage is for. |
| `RUNTIME` | The running process needs it. | No. |
| `DATA` | Weights and assets. Needed at run time. | Not dropped — **relocated** to a volume or object store. |

The distinction between `BUILD` and `RUNTIME` is what makes multi-stage
possible. The distinction between `RUNTIME` and `DATA` is what makes the bigger
saving possible, and it is the one people do not draw.

Note that `cuda-devel` is `BUILD` and `cuda-runtime` is `RUNTIME`. Confusing
them is the most common way to produce either a broken image or a needlessly
enormous one.

examples-run.txt

--- 1. single stage, full base, weights baked in ---
  image  21128 MB   one pull   845.1s   rollout to 20 nodes  16902.0s
  [build-tooling-shipped] saves 4380 MB
      build-only components in the final image (build-essential, cuda-devel); produc
  [weights-baked-in] saves 13500 MB
      model weights are in the image (13500 MB); every node pulls them on every roll
  [fat-base] saves 295 MB
      the final stage uses the full Python base; the slim base carries the same inte
--- 2. multi-stage, slim runtime, weights baked in ---
  image  18053 MB   one pull   722.1s   rollout to 20 nodes  14442.0s
  [weights-baked-in] saves 13500 MB
      model weights are in the image (13500 MB); every node pulls them on every roll
--- 3. multi-stage, slim runtime, weights mounted ---
  image   4553 MB   one pull   182.1s   rollout to 20 nodes   3642.0s
  no findings
--- what each change bought ---
  multi-stage       21128 ->  18053 MB  (3075 MB, 1.17x)
  mount the weights 18053 ->   4553 MB  (13500 MB, 3.97x)
  both              21128 ->   4553 MB  (16575 MB, 4.64x)
  rollout to 20 nodes: 16902s -> 3642s

measured-values.txt

Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.

  python3 examples/image_demo.py -> 20 lines, exit 0
  bash tests/run_tests.sh        -> 20 passed in 0.01s
  the untouched starter          -> 20 failed in 0.06s

The same AI service packaged three ways:

  1. single stage, full base, weights baked in    21128 MB
  2. multi-stage, slim runtime, weights baked in  18053 MB
  3. multi-stage, slim runtime, weights mounted    4553 MB

What each change bought:

  multi-stage        21128 ->  18053 MB   3075 MB saved   1.17x
  mount the weights  18053 ->   4553 MB  13500 MB saved   3.97x
  both               21128 ->   4553 MB  16575 MB saved   4.64x

This is the number this lab exists for, and it is not the one people expect.
The multi-stage build is the optimisation every Docker tutorial teaches, and
here it is the SMALLER win by a factor of four. Moving the weights out of the
image saves 13.5 GB; removing the build toolchain and the fat base saves 3.1.

The reason is simply that most Docker guidance was written for web
applications, where the largest thing in the image is the framework. In an AI
image the largest thing is usually the model, and no amount of multi-stage
discipline touches it.

Where the size is actually felt:

  rollout to 20 nodes   16902s  ->  3642s

Four hours and 42 minutes down to one hour and one minute, for the same
service. A rollout pays the pull once per node, which is why image size is a
deployment-time property rather than a disk-space one.

One implementation note. cuda-devel (4100 MB) is BUILD and cuda-runtime
(1600 MB) is RUNTIME. Treating them as interchangeable produces either a
broken image, if you drop the runtime, or a needlessly enormous one, if you
keep the devel package. The review function relies on that distinction and so
does the whole multi-stage argument.

A second note on the test suite. An earlier version of
test_moving_the_weights_out_beats_the_multi_stage_build asserted that mounting
saves more than 3x what multi-stage saves. That failed against a fixture whose
single-stage image carried cuda-devel, where the multi-stage saving is unusually
large -- 4675 MB against 13500. The claim that actually holds is the ORDERING,
not a fixed ratio, and the test now asserts that.

run-output.txt

--- 1. single stage, full base, weights baked in ---
  image  21128 MB   one pull   845.1s   rollout to 20 nodes  16902.0s
  [build-tooling-shipped] saves 4380 MB
      build-only components in the final image (build-essential, cuda-devel); produc
  [weights-baked-in] saves 13500 MB
      model weights are in the image (13500 MB); every node pulls them on every roll
  [fat-base] saves 295 MB
      the final stage uses the full Python base; the slim base carries the same inte
--- 2. multi-stage, slim runtime, weights baked in ---
  image  18053 MB   one pull   722.1s   rollout to 20 nodes  14442.0s
  [weights-baked-in] saves 13500 MB
      model weights are in the image (13500 MB); every node pulls them on every roll
--- 3. multi-stage, slim runtime, weights mounted ---
  image   4553 MB   one pull   182.1s   rollout to 20 nodes   3642.0s
  no findings
--- what each change bought ---
  multi-stage       21128 ->  18053 MB  (3075 MB, 1.17x)
  mount the weights 18053 ->   4553 MB  (13500 MB, 3.97x)
  both              21128 ->   4553 MB  (16575 MB, 4.64x)
  rollout to 20 nodes: 16902s -> 3642s

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

FAILED tests/test_image_plan.py::test_a_rollout_to_no_nodes_costs_nothing - N...
FAILED tests/test_image_plan.py::test_build_tooling_in_the_final_image_is_reported
FAILED tests/test_image_plan.py::test_build_tooling_confined_to_an_earlier_stage_is_not_reported
FAILED tests/test_image_plan.py::test_baked_in_weights_are_reported_with_what_they_cost
FAILED tests/test_image_plan.py::test_mounted_weights_produce_no_finding - No...
FAILED tests/test_image_plan.py::test_a_full_base_in_the_final_stage_is_reported
FAILED tests/test_image_plan.py::test_a_clean_plan_has_no_findings - NotImple...
FAILED tests/test_image_plan.py::test_carried_build_tooling_is_caught_too - N...
FAILED tests/test_image_plan.py::test_compare_reports_what_was_saved_and_the_ratio
FAILED tests/test_image_plan.py::test_comparing_a_plan_with_itself_saves_nothing
FAILED tests/test_image_plan.py::test_moving_the_weights_out_beats_the_multi_stage_build
20 failed in 0.06s

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

....................                                                     [100%]
20 passed in 0.01s

test-run.txt

....................                                                     [100%]
20 passed in 0.01s

Source files

examples/image_demo.py (3259 bytes)
#!/usr/bin/env python3
"""The same AI service packaged three ways."""

from __future__ import annotations

import os
import sys

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

from image_plan import (
    Component,
    ImagePlan,
    Purpose,
    Stage,
    compare,
    pull_seconds,
    review,
    rollout_seconds,
)

NODES = 20

RUNTIME = [
    Component("torch", Purpose.RUNTIME),
    Component("transformers", Purpose.RUNTIME),
    Component("fastapi", Purpose.RUNTIME),
    Component("app-code", Purpose.RUNTIME),
]

# 1. One stage, full base, build tools and weights all in the shipped image.
NAIVE = ImagePlan(
    stages=[
        Stage("final", "python-full", [
            Component("build-essential", Purpose.BUILD),
            Component("cuda-devel", Purpose.BUILD),
            *RUNTIME,
            Component("model-weights-7b", Purpose.DATA),
        ])
    ],
    final_stage="final",
)

# 2. Multi-stage: compile in a builder, ship a slim runtime. Weights still in.
STAGED = ImagePlan(
    stages=[
        Stage("builder", "python-full", [
            Component("build-essential", Purpose.BUILD),
            Component("cuda-devel", Purpose.BUILD),
        ]),
        Stage("final", "python-slim", [
            Component("cuda-runtime", Purpose.RUNTIME),
            *RUNTIME,
            Component("model-weights-7b", Purpose.DATA),
        ]),
    ],
    final_stage="final",
)

# 3. Same as 2, with the weights mounted at run time instead of baked in.
MOUNTED = ImagePlan(
    stages=[
        Stage("builder", "python-full", [
            Component("build-essential", Purpose.BUILD),
            Component("cuda-devel", Purpose.BUILD),
        ]),
        Stage("final", "python-slim", [
            Component("cuda-runtime", Purpose.RUNTIME),
            *RUNTIME,
        ]),
    ],
    final_stage="final",
)


def show(name: str, plan: ImagePlan) -> None:
    print(f"--- {name} ---")
    print(f"  image {plan.size_mb:>6} MB   one pull {pull_seconds(plan.size_mb):>7.1f}s"
          f"   rollout to {NODES} nodes {rollout_seconds(plan.size_mb, NODES):>8.1f}s")
    found = review(plan)
    for f in found:
        print(f"  [{f.rule}] saves {f.saves_mb} MB")
        print(f"      {f.message[:78]}")
    if not found:
        print("  no findings")


def main() -> int:
    show("1. single stage, full base, weights baked in", NAIVE)
    show("2. multi-stage, slim runtime, weights baked in", STAGED)
    show("3. multi-stage, slim runtime, weights mounted", MOUNTED)

    print("--- what each change bought ---")
    a = compare(NAIVE, STAGED, nodes=NODES)
    print(f"  multi-stage      {a['before_mb']:>6} -> {a['after_mb']:>6} MB  "
          f"({a['saved_mb']} MB, {a['ratio']}x)")
    b = compare(STAGED, MOUNTED, nodes=NODES)
    print(f"  mount the weights{b['before_mb']:>6} -> {b['after_mb']:>6} MB  "
          f"({b['saved_mb']} MB, {b['ratio']}x)")
    c = compare(NAIVE, MOUNTED, nodes=NODES)
    print(f"  both             {c['before_mb']:>6} -> {c['after_mb']:>6} MB  "
          f"({c['saved_mb']} MB, {c['ratio']}x)")
    print(f"  rollout to {NODES} nodes: {c['before_rollout_s']:.0f}s -> {c['after_rollout_s']:.0f}s")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
examples/image_plan.py (5784 bytes)
"""Work out how big an AI image will be, and what is making it big.

Offline and standard-library only. Docker is not required: the input is a
described build -- stages, what each installs, and what the final stage keeps
-- so the arithmetic can be checked without pulling six gigabytes to find out.

Packaging an AI application is not packaging a web app with a bigger
requirements file. Three things dominate, and only one of them is obvious:

  build tooling   compilers and headers needed to BUILD wheels, never to run
  the framework   torch and CUDA runtime are gigabytes before your code exists
  model weights   frequently larger than everything else combined

The first is solved by a multi-stage build. The third is usually solved by NOT
solving it -- weights baked into an image are pulled onto every node on every
rollout, and a volume or object store is almost always the right answer.

Nothing here judges whether an image works. It computes what it costs to pull,
which is the number that decides how long a deploy takes.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum

# Nominal compressed sizes in megabytes. The absolute values are illustrative;
# the ORDER OF MAGNITUDE between them is the part that drives every decision.
COMPONENT_MB = {
    "python-slim": 45,
    "python-full": 340,
    "build-essential": 280,     # gcc, make, headers -- build only
    "cuda-runtime": 1_600,
    "cuda-devel": 4_100,        # build only
    "torch": 2_400,
    "transformers": 480,
    "fastapi": 25,
    "app-code": 3,
    "model-weights-7b": 13_500,
    "model-weights-small": 420,
}


class Purpose(str, Enum):
    """Why a component is present, which decides whether it may be dropped."""

    BUILD = "build"      # needed to produce artefacts, never at run time
    RUNTIME = "runtime"  # needed by the running process
    DATA = "data"        # weights and assets -- runtime, but relocatable


@dataclass(frozen=True)
class Component:
    name: str
    purpose: Purpose

    @property
    def size_mb(self) -> int:
        return COMPONENT_MB.get(self.name, 0)


@dataclass
class Stage:
    name: str
    base: str
    components: list[Component] = field(default_factory=list)

    @property
    def size_mb(self) -> int:
        return COMPONENT_MB.get(self.base, 0) + sum(c.size_mb for c in self.components)


@dataclass
class ImagePlan:
    """A build described as stages plus what the final stage carries forward."""

    stages: list[Stage]
    final_stage: str
    carried: list[Component] = field(default_factory=list)

    @property
    def final(self) -> Stage:
        for s in self.stages:
            if s.name == self.final_stage:
                return s
        raise KeyError(f"no stage named {self.final_stage!r}")

    @property
    def size_mb(self) -> int:
        """Only the final stage ships. Earlier stages are discarded."""
        return self.final.size_mb + sum(c.size_mb for c in self.carried)

    @property
    def build_only_shipped(self) -> list[Component]:
        """Build tooling that reached the final image. Pure waste."""
        present = self.final.components + self.carried
        return [c for c in present if c.purpose is Purpose.BUILD]

    @property
    def weights_shipped(self) -> list[Component]:
        present = self.final.components + self.carried
        return [c for c in present if c.purpose is Purpose.DATA]


def pull_seconds(size_mb: int, mbps: float = 200.0) -> float:
    """How long one node waits to pull this image, at a given link speed."""
    if mbps <= 0:
        return 0.0
    return round(size_mb * 8 / mbps, 1)


def rollout_seconds(size_mb: int, nodes: int, mbps: float = 200.0) -> float:
    """A rollout pulls onto every node. This is where image size is felt."""
    return round(pull_seconds(size_mb, mbps) * max(0, nodes), 1)


@dataclass(frozen=True)
class Finding:
    rule: str
    message: str
    saves_mb: int = 0


def review(plan: ImagePlan) -> list[Finding]:
    """What is making this image big, and what removing it would save."""
    out: list[Finding] = []

    build = plan.build_only_shipped
    if build:
        waste = sum(c.size_mb for c in build)
        names = ", ".join(c.name for c in build)
        out.append(
            Finding(
                "build-tooling-shipped",
                f"build-only components in the final image ({names}); "
                "produce them in an earlier stage and copy the artefacts out",
                waste,
            )
        )

    weights = plan.weights_shipped
    if weights:
        waste = sum(c.size_mb for c in weights)
        out.append(
            Finding(
                "weights-baked-in",
                f"model weights are in the image ({sum(c.size_mb for c in weights)} MB); "
                "every node pulls them on every rollout even when only your code changed",
                waste,
            )
        )

    if plan.final.base == "python-full":
        out.append(
            Finding(
                "fat-base",
                "the final stage uses the full Python base; the slim base carries the same "
                "interpreter without the build toolchain",
                COMPONENT_MB["python-full"] - COMPONENT_MB["python-slim"],
            )
        )
    return out


def compare(before: ImagePlan, after: ImagePlan, *, nodes: int = 20) -> dict[str, float]:
    """What the change bought, in the units that are actually felt."""
    b, a = before.size_mb, after.size_mb
    return {
        "before_mb": b,
        "after_mb": a,
        "saved_mb": b - a,
        "ratio": round(b / a, 2) if a else 0.0,
        "before_rollout_s": rollout_seconds(b, nodes),
        "after_rollout_s": rollout_seconds(a, nodes),
    }
metadata.yml (929 bytes)
lesson_id: D331
day: 331
kind: python
languages:
  - python
setup_commands:
  - pip install -r requirements/requirements.txt
run_commands:
  - python3 examples/image_demo.py
test_commands:
  - bash tests/run_tests.sh
cleanup_commands:
  - rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3 examples/image_demo.py
  reproduced the documented figures exactly (21128 MB -> 18053 MB -> 4553 MB; multi-stage 1.17x,
  moving the weights out 3.97x, both together 4.64x; a 20-node rollout falling from 16902s to 3642s);
  bash tests/run_tests.sh -> 20 passed in 0.01s. The untouched starter fails all 20. requires_network
  is false and Docker is NOT required: the input is a described build, so the arithmetic is checkable
  without pulling twenty gigabytes to find out.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/image_plan.py (4372 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.

Work out how big an AI image will be, and what is making it big.

Offline and standard-library only. Docker is not required: the input is a
described build -- stages, what each installs, and what the final stage keeps
-- so the arithmetic can be checked without pulling six gigabytes to find out.

Packaging an AI application is not packaging a web app with a bigger
requirements file. Three things dominate, and only one of them is obvious:

  build tooling   compilers and headers needed to BUILD wheels, never to run
  the framework   torch and CUDA runtime are gigabytes before your code exists
  model weights   frequently larger than everything else combined

The first is solved by a multi-stage build. The third is usually solved by NOT
solving it -- weights baked into an image are pulled onto every node on every
rollout, and a volume or object store is almost always the right answer.

Nothing here judges whether an image works. It computes what it costs to pull,
which is the number that decides how long a deploy takes.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
COMPONENT_MB = {'python-slim': 45, 'python-full': 340, 'build-essential': 280, 'cuda-runtime': 1600, 'cuda-devel': 4100, 'torch': 2400, 'transformers': 480, 'fastapi': 25, 'app-code': 3, 'model-weights-7b': 13500, 'model-weights-small': 420}

class Purpose(str, Enum):
    """Why a component is present, which decides whether it may be dropped."""
    BUILD = 'build'
    RUNTIME = 'runtime'
    DATA = 'data'

@dataclass(frozen=True)
class Component:
    name: str
    purpose: Purpose

    @property
    def size_mb(self) -> int:
        raise NotImplementedError('TASK 1: implement size_mb.')

@dataclass
class Stage:
    name: str
    base: str
    components: list[Component] = field(default_factory=list)

    @property
    def size_mb(self) -> int:
        raise NotImplementedError('TASK 2: implement size_mb.')

@dataclass
class ImagePlan:
    """A build described as stages plus what the final stage carries forward."""
    stages: list[Stage]
    final_stage: str
    carried: list[Component] = field(default_factory=list)

    @property
    def final(self) -> Stage:
        raise NotImplementedError('TASK 3: implement final.')

    @property
    def size_mb(self) -> int:
        """Only the final stage ships. Earlier stages are discarded."""
        raise NotImplementedError('TASK 4: implement size_mb. Only the final stage ships. Earlier stages are discarded.')

    @property
    def build_only_shipped(self) -> list[Component]:
        """Build tooling that reached the final image. Pure waste."""
        raise NotImplementedError('TASK 5: implement build_only_shipped. Build tooling that reached the final image. Pure waste.')

    @property
    def weights_shipped(self) -> list[Component]:
        raise NotImplementedError('TASK 6: implement weights_shipped.')

def pull_seconds(size_mb: int, mbps: float=200.0) -> float:
    """How long one node waits to pull this image, at a given link speed."""
    raise NotImplementedError('TASK 7: implement pull_seconds. How long one node waits to pull this image, at a given link speed.')

def rollout_seconds(size_mb: int, nodes: int, mbps: float=200.0) -> float:
    """A rollout pulls onto every node. This is where image size is felt."""
    raise NotImplementedError('TASK 8: implement rollout_seconds. A rollout pulls onto every node. This is where image size is felt.')

@dataclass(frozen=True)
class Finding:
    rule: str
    message: str
    saves_mb: int = 0

def review(plan: ImagePlan) -> list[Finding]:
    """What is making this image big, and what removing it would save."""
    raise NotImplementedError('TASK 9: implement review. What is making this image big, and what removing it would save.')

def compare(before: ImagePlan, after: ImagePlan, *, nodes: int=20) -> dict[str, float]:
    """What the change bought, in the units that are actually felt."""
    raise NotImplementedError('TASK 10: implement compare. What the change bought, in the units that are actually felt.')
tests/run_tests.sh (200 bytes)
#!/usr/bin/env bash
# Run the day 331 image-plan test suite.
# Offline, standard library plus pytest. Docker is not required.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_image_plan.py (7025 bytes)
"""Grouped by what the image plan decides.

Run with: bash tests/run_tests.sh
"""

from __future__ import annotations

import os
import sys

HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "examples"))

from image_plan import (  # noqa: E402
    COMPONENT_MB,
    Component,
    ImagePlan,
    Purpose,
    Stage,
    compare,
    pull_seconds,
    review,
    rollout_seconds,
)


def rt(*names):
    return [Component(n, Purpose.RUNTIME) for n in names]


def single(base="python-slim", comps=None):
    return ImagePlan(stages=[Stage("final", base, comps or [])], final_stage="final")


# ------------------------------------------------------------------ sizing


def test_a_stage_is_its_base_plus_its_components():
    s = Stage("final", "python-slim", rt("fastapi", "app-code"))
    assert s.size_mb == COMPONENT_MB["python-slim"] + COMPONENT_MB["fastapi"] + COMPONENT_MB["app-code"]


def test_an_unknown_component_contributes_nothing_rather_than_raising():
    assert Component("not-a-real-package", Purpose.RUNTIME).size_mb == 0


def test_only_the_final_stage_ships():
    # THE point of a multi-stage build: the builder is discarded entirely.
    plan = ImagePlan(
        stages=[
            Stage("builder", "python-full", [Component("cuda-devel", Purpose.BUILD)]),
            Stage("final", "python-slim", rt("app-code")),
        ],
        final_stage="final",
    )
    assert plan.size_mb == COMPONENT_MB["python-slim"] + COMPONENT_MB["app-code"]
    assert COMPONENT_MB["cuda-devel"] not in (plan.size_mb,)


def test_carried_components_do_count_toward_the_final_size():
    plan = ImagePlan(
        stages=[Stage("final", "python-slim", [])],
        final_stage="final",
        carried=rt("torch"),
    )
    assert plan.size_mb == COMPONENT_MB["python-slim"] + COMPONENT_MB["torch"]


def test_naming_a_stage_that_does_not_exist_is_an_error():
    plan = ImagePlan(stages=[Stage("final", "python-slim", [])], final_stage="typo")
    try:
        plan.size_mb
    except KeyError:
        return
    raise AssertionError("expected a KeyError for an unknown final stage")


# -------------------------------------------------------------- pull cost


def test_pull_time_scales_with_size():
    assert pull_seconds(200, mbps=200.0) == 8.0
    assert pull_seconds(400, mbps=200.0) == 16.0


def test_a_faster_link_pulls_proportionally_faster():
    assert pull_seconds(1000, mbps=100.0) == 2 * pull_seconds(1000, mbps=200.0)


def test_a_nonsensical_link_speed_does_not_divide_by_zero():
    assert pull_seconds(1000, mbps=0.0) == 0.0


def test_a_rollout_pays_the_pull_once_per_node():
    assert rollout_seconds(1000, 10) == 10 * pull_seconds(1000)


def test_a_rollout_to_no_nodes_costs_nothing():
    assert rollout_seconds(1000, 0) == 0.0
    assert rollout_seconds(1000, -5) == 0.0


# ------------------------------------------------------------------ review


def test_build_tooling_in_the_final_image_is_reported():
    plan = single(comps=[Component("cuda-devel", Purpose.BUILD), *rt("app-code")])
    f = next(x for x in review(plan) if x.rule == "build-tooling-shipped")
    assert f.saves_mb == COMPONENT_MB["cuda-devel"]


def test_build_tooling_confined_to_an_earlier_stage_is_not_reported():
    plan = ImagePlan(
        stages=[
            Stage("builder", "python-full", [Component("cuda-devel", Purpose.BUILD)]),
            Stage("final", "python-slim", rt("app-code")),
        ],
        final_stage="final",
    )
    assert "build-tooling-shipped" not in {f.rule for f in review(plan)}


def test_baked_in_weights_are_reported_with_what_they_cost():
    plan = single(comps=[Component("model-weights-7b", Purpose.DATA), *rt("app-code")])
    f = next(x for x in review(plan) if x.rule == "weights-baked-in")
    assert f.saves_mb == COMPONENT_MB["model-weights-7b"]


def test_mounted_weights_produce_no_finding():
    assert "weights-baked-in" not in {f.rule for f in review(single(comps=rt("app-code")))}


def test_a_full_base_in_the_final_stage_is_reported():
    f = next(x for x in review(single(base="python-full", comps=rt("app-code"))) if x.rule == "fat-base")
    assert f.saves_mb == COMPONENT_MB["python-full"] - COMPONENT_MB["python-slim"]


def test_a_clean_plan_has_no_findings():
    assert review(single(comps=rt("cuda-runtime", "torch", "fastapi", "app-code"))) == []


def test_carried_build_tooling_is_caught_too():
    # Copying a build artefact forward is fine; copying the toolchain is not.
    plan = ImagePlan(
        stages=[Stage("final", "python-slim", rt("app-code"))],
        final_stage="final",
        carried=[Component("build-essential", Purpose.BUILD)],
    )
    assert "build-tooling-shipped" in {f.rule for f in review(plan)}


# ----------------------------------------------------------------- compare


def test_compare_reports_what_was_saved_and_the_ratio():
    before = single(comps=[Component("model-weights-7b", Purpose.DATA), *rt("app-code")])
    after = single(comps=rt("app-code"))
    c = compare(before, after)
    assert c["saved_mb"] == COMPONENT_MB["model-weights-7b"]
    assert c["ratio"] > 1.0


def test_comparing_a_plan_with_itself_saves_nothing():
    p = single(comps=rt("app-code"))
    c = compare(p, p)
    assert c["saved_mb"] == 0 and c["ratio"] == 1.0


def test_moving_the_weights_out_beats_the_multi_stage_build():
    # The finding this lab exists for: multi-stage is the famous optimisation
    # and it is the SMALLER one for an AI image.
    one_stage = single(
        base="python-full",
        comps=[
            Component("build-essential", Purpose.BUILD),
            Component("cuda-devel", Purpose.BUILD),
            *rt("cuda-runtime", "torch", "app-code"),
            Component("model-weights-7b", Purpose.DATA),
        ],
    )
    staged = ImagePlan(
        stages=[
            Stage("builder", "python-full", [
                Component("build-essential", Purpose.BUILD),
                Component("cuda-devel", Purpose.BUILD),
            ]),
            Stage("final", "python-slim", [
                *rt("cuda-runtime", "torch", "app-code"),
                Component("model-weights-7b", Purpose.DATA),
            ]),
        ],
        final_stage="final",
    )
    mounted = ImagePlan(
        stages=staged.stages[:1] + [Stage("final", "python-slim", rt("cuda-runtime", "torch", "app-code"))],
        final_stage="final",
    )
    from_multistage = compare(one_stage, staged)["saved_mb"]
    from_mounting = compare(staged, mounted)["saved_mb"]

    # Moving the weights out is the bigger single win. How much bigger depends
    # on how much build tooling the one-stage image carried, so the claim
    # tested here is the ordering, not a fixed ratio -- an earlier version of
    # this test asserted 3x and failed on a fixture that shipped cuda-devel,
    # where the multi-stage saving is unusually large.
    assert from_mounting > from_multistage
    assert from_mounting == COMPONENT_MB["model-weights-7b"]

Troubleshooting

Troubleshooting — Day 331

NotImplementedError on every test

Expected. The starter stubs ten functions — see expected-output/starter-run.txt.

Start with Stage.size_mb; nothing else can be checked until a stage has a size.

test_only_the_final_stage_ships fails

The heart of a multi-stage build: earlier stages are discarded entirely. ImagePlan.size_mb must consult only the final stage plus whatever was carried into it — if you are summing every stage, the builder's four gigabytes are still in your total and multi-stage appears to buy nothing.

test_naming_a_stage_that_does_not_exist_is_an_error fails

Returning None or falling back to the first stage hides a typo in final_stage and produces a confidently wrong number. Raise KeyError.

An unknown component crashes instead of contributing zero

COMPONENT_MB.get(name, 0), not COMPONENT_MB[name]. A plan naming a package the table does not know about is a normal thing to write, and it should not stop the arithmetic.

pull_seconds(size, mbps=0) must return 0.0 rather than raising. Zero bandwidth is nonsense input, and a tool that crashes on it is harder to use than one that returns a harmless value.

test_a_rollout_to_no_nodes_costs_nothing fails

Guard the node count with max(0, nodes). A negative count should not produce negative seconds.

build-tooling-shipped misses a carried component

review must look at the final stage's components and the carried list. Copying a build artefact forward is the point of a multi-stage build; copying the toolchain forward defeats it, and that is exactly the case worth catching.

test_a_clean_plan_has_no_findings fails

Something fires on a good plan. The usual cause is treating cuda-runtime as build tooling — it is RUNTIME, and the runtime is what a GPU service needs. Only cuda-devel is build-only.

The multi-stage saving looks bigger than the weights saving

That can be legitimate, and it is worth understanding rather than "fixing". If the single-stage image carried cuda-devel (about 4 GB) then removing it is a large win. The lab's demo does not, which is why mounting the weights dominates there. The test asserts the ordering that holds for the demo's shapes, not a fixed ratio.

Security notes

Security notes — Day 331

What this lab does and does not touch

  • No Docker required and none is invoked. The input is a described build. Nothing is pulled, built or run.
  • No network, no credentials, no registry login.
  • No writes outside this directory. Only __pycache__/, .pytest_cache/ and, if you create it, .venv/.
  • No sudo, no global installs.
  • No model weights are downloaded. The 13.5 GB in the demo is a number in a table, which is the point — you should not need to move that much data to reason about whether you should.

A smaller image is a smaller attack surface

The size argument and the security argument point the same way, which is unusual and worth using.

Build tooling in a runtime image is a compiler in production. cuda-devel, gcc, make and the headers exist to produce artefacts. Left in the final image they give anyone who achieves code execution a full toolchain to build with. A multi-stage build removes them for a reason that has nothing to do with megabytes.

A slim base has fewer packages to have vulnerabilities. Most CVEs reported against an application image are in components the application never calls. The cheapest way to stop being affected is not to ship them.

Every layer is kept — including the one with the weights. If a model is licensed such that you may not redistribute it, baking it into an image you push to a registry is redistribution, and pushing to a public registry is publication. This is a licence question that looks like an engineering decision.

Weights deserve their own answer

The demo's third plan mounts the weights rather than baking them in. Three consequences beyond size:

  • The mount is now a supply-chain input. Whatever writes to that volume can replace your model. Day 347 covers this properly; the short version is that a checksum on load is cheap and worth it.
  • A public image no longer contains a private model. Separating them means an image leak is not a model leak.
  • Rotating the model no longer requires rebuilding the image, which removes the temptation to rebuild in a hurry without re-running the checks.

Run as a non-root user, and mean it

Day 330's linter checks for USER. It matters more here: an AI service typically has a writable cache directory, a mounted model volume and a network listener. Root inside the container plus a writable mount is the combination that turns a container escape from theoretical into practical.

Reversing everything this lab did

rm -rf __pycache__ .pytest_cache .venv

Nothing else was created, and nothing outside this directory was modified.