Deployment, MLOps, and Security › Containers and Cloud › Day 332

Hands-on lab — Day 332: Docker Compose for Multi-Service Apps

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/compose_demo.py

Test

bash tests/run_tests.sh

File tree

examples/compose_demo.py
examples/compose_graph.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/compose_graph.py
tests/run_tests.sh
tests/test_compose_graph.py
troubleshooting.md

Lab README

Day 332 Lab: Docker Compose for Multi-Service Apps

Lesson

Purpose

Work out what a Compose file will actually do before you run it.

A multi-service application is a dependency graph, and three things go wrong in it that are all visible before anything starts: cycles, host-port clashes, and false ordering. The third is the one that bites — depends_on waits for the container, not for the service, so an API can be "up" four seconds before its database can answer a query.

Learning objectives

  • Detect dependency cycles, reporting each once rather than once per member.
  • Produce a stable startup order, and explain why a cycle has none.
  • Distinguish a host-port conflict from two containers sharing an internal port.
  • Model what service_started and service_healthy actually wait for.
  • Explain why the correctly wired stack is slower to report ready, and why that is the honest number.

Prerequisites

  • Day 331, "Dockerizing an AI Application".
  • Comfortable with Python dataclasses, enums and basic graph traversal.

Supported operating systems

macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.

Hardware requirements

Any machine running Python 3.10 or newer. No container runtime. The suite completes in hundredths of a second.

Required software

  • Python 3.10 or newer.
  • pytest 7 or newer.

Docker Compose is not required. The questions worth asking about a Compose file are graph questions, and a graph question is answerable before anything runs — which is the whole argument for asking them.

Free and open-source options

Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. For the real thing, Docker Compose is Apache-2.0, and podman-compose and podman kube play read the same files without a daemon.

Installation

python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt

Removing .venv fully reverses the installation.

File structure

starter/compose_graph.py     your work: nine stubbed tasks
examples/compose_graph.py    reference implementation
examples/compose_demo.py     one AI stack wired two ways, plus a broken file
tests/test_compose_graph.py  grouped by what the check decides
tests/run_tests.sh           suite entry point
expected-output/             real captured output and measured values
requirements/                pinned dependency

How to run

python3 examples/compose_demo.py   # compare two wirings and inspect a broken file
bash tests/run_tests.sh            # run the suite

To work on the exercise, edit starter/compose_graph.py, then copy it over the reference:

cp starter/compose_graph.py examples/compose_graph.py
bash tests/run_tests.sh

Keep a copy of the reference first if you want to compare.

What the commands do

python3 examples/compose_demo.py takes a four-service AI stack — postgres, qdrant, api, worker — wires it first with bare depends_on and then with service_healthy, prints the start order and when each service is genuinely usable, and reports every finding. It then runs a deliberately broken file containing a cycle, an undefined dependency and a port clash.

bash tests/run_tests.sh runs pytest over twenty-six tests grouped by what the check decides: cycles, undefined dependencies, ports, ordering, readiness, premature starts and the combined review.

Expected output

--- what the condition changed ---
  api usable at    1.2s  ->  5.2s
  worker usable at 1.2s  ->  5.8s
  premature starts 4  ->  0

The correct stack is slower, and that is the point: at 1.2s the api was not usable, it was merely running, and its first query would have failed.

Validation steps

  1. bash tests/run_tests.sh reports 26 passed.
  2. Under service_started the api must be usable before postgres is. If the two stacks behave identically, ready_times is treating service_started as though it waited for readiness.
  3. A three-service cycle must be reported once, not three times.
  4. 8000:8080 and 8001:8080 must not conflict. Only the host side can.
  5. A cycle must suppress the timing findings — there is no start order to reason about.

Tests

Twenty-six tests in seven groups:

  • cycles — simple, longer and self-dependencies; reported once; a diamond is not a cycle.
  • undefined dependencies — reported, and defined ones are not.
  • ports — host clashes conflict; the same container port on different host ports does not.
  • order — dependencies precede dependants; the order is stable; a cycle raises; an undefined dependency does not block.
  • readiness — a leaf is start plus ready; service_started does not wait; service_healthy does; waiting is slower.
  • premature starts — reported with the gap in seconds; absent under a healthy condition; absent when the dependency is instant.
  • review — every class found; a cycle suppresses timing; service_healthy on a service with no healthcheck is reported; a correct stack is clean.

Cleanup

rm -rf __pycache__ .pytest_cache
rm -rf .venv

Both are safe to remove and fully reversible.

Troubleshooting

See troubleshooting.md.

Security notes

See security.md. In short: publishing a port binds it on the host, which on a cloud VM can mean the internet — and services on a Compose network already reach each other without it.

Extension exercises

  1. Add restart policies and retry. Model restart: on-failure and a client that retries with backoff. Then answer the question the lesson raises: does retry make service_healthy unnecessary? Measure the failed requests during startup under each combination before deciding.
  2. Parse a real docker-compose.yml. Read one with yaml.safe_load, build the Service graph from it, and run the review over a stack you actually operate.
  3. Add resource limits. Give each service a memory reservation and report when the declared total exceeds what the host has — the failure mode being that Compose starts everything and the kernel then kills whichever container asks for memory last.
  • Lesson
  • Previous: Day 331 — Dockerizing an AI Application
  • Next: Day 333 — Kubernetes Concepts

Expected output

FIELDS.md

# Reading the compose graph report

## The start order

    start order  postgres -> qdrant -> api -> worker

A topological order of the dependency graph, ties broken by name so the result
is stable. It says what starts before what. It says nothing about when anything
is *usable*, which is the next line and the whole point.

## The readiness line

    usable at    api 1.2s  postgres 4.4s  qdrant 2.3s  worker 1.2s

When each service can actually serve. Read `api` against `postgres`:

    api      usable at 1.2s
    postgres usable at 4.4s

The api is up three seconds before the database it queries can answer. That is
not a bug in the model — it is exactly what `depends_on` without a condition
means, and the report exists to make it visible.

## The findings

| Rule | Severity | Meaning |
| --- | --- | --- |
| `dependency-cycle` | fatal | Nothing in the cycle can ever start. Suppresses the timing findings, because there is no order. |
| `undefined-dependency` | error | A `depends_on` naming a service the file does not define. |
| `port-conflict` | error | Two services binding the same **host** port. |
| `starts-before-ready` | warning | A dependant begins before its dependency can answer. |
| `healthy-without-healthcheck` | error | Something waits for `service_healthy` on a service that defines no healthcheck, so the condition can never be satisfied. |

## The three conditions

| Condition | Waits for | When to use it |
| --- | --- | --- |
| `service_started` | The container exists. | Almost never on its own — this is the default and the source of the problem. |
| `service_healthy` | The healthcheck passed. | Anything a dependant will immediately query. |
| `service_completed_successfully` | A one-shot job exited zero. | Migrations, seeding, index building. |

## What the condition changed

    api usable at    1.2s  ->  5.2s
    worker usable at 1.2s  ->  5.8s
    premature starts 4  ->  0

**The correct stack is slower.** That is not a cost to be weighed — at 1.2s the
api was not usable, it was merely running, and any query it issued would have
failed. The second column is the first honest measurement of when the stack
works.

This is the reverse of the usual optimisation story, and it is why
`test_waiting_properly_is_slower_and_that_is_the_point` exists: a student who
"improves" the numbers has broken the lesson.

## Only the host side of a port can conflict

    port-conflict  host port 8000 is claimed by 2 services: api, worker

Two containers may both listen on 8080 internally; they are in separate network
namespaces. `ports: ["8000:8080"]` binds 8000 on the host, and the host has one
of each.

examples-run.txt

--- depends_on with no condition ---
  start order  postgres -> qdrant -> api -> worker
  usable at    api 1.2s  postgres 4.4s  qdrant 2.3s  worker 1.2s
  [starts-before-ready] api
      starts 4.0s before 'postgres' can answer; depends_on without a condition wai
  [starts-before-ready] api
      starts 1.9s before 'qdrant' can answer; depends_on without a condition waits
  [starts-before-ready] worker
      starts 3.8s before 'postgres' can answer; depends_on without a condition wai
  [starts-before-ready] worker
      starts 0.6s before 'api' can answer; depends_on without a condition waits fo
--- depends_on: service_healthy ---
  start order  postgres -> qdrant -> api -> worker
  usable at    api 5.2s  postgres 4.4s  qdrant 2.3s  worker 5.8s
  no findings
--- a file that cannot start ---
  cannot start: dependency cycle: api -> worker -> api
  [undefined-dependency] ghost
      depends on 'nowhere', which no service defines
  [port-conflict] api, worker
      host port 8000 is claimed by 2 services: api, worker
  [dependency-cycle] api
      cycle: api -> worker -> api; nothing in it can start
--- what the condition changed ---
  api usable at    1.2s  ->  5.2s
  worker usable at 1.2s  ->  5.8s
  premature starts 4  ->  0
  cycles in the broken file: [['api', 'worker']]

measured-values.txt

Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.

  python3 examples/compose_demo.py -> 28 lines, exit 0
  bash tests/run_tests.sh          -> 26 passed in 0.01s
  the untouched starter            -> 26 failed in 0.08s

One AI stack -- postgres, qdrant, api, worker -- wired two ways. The only
difference is the depends_on condition and the presence of healthchecks.

  depends_on with no condition   api usable 1.2s   worker usable 1.2s   4 premature starts
  depends_on: service_healthy    api usable 5.2s   worker usable 5.8s   0 premature starts

The naive stack is FASTER and wrong. Read it against the dependency:

  api      usable at 1.2s
  postgres usable at 4.4s

The api reports itself up three and a bit seconds before the database it
queries can accept a connection. Its first query fails. On a laptop, where
postgres is warm and the page cache is full, it usually does not; in CI, where
everything is cold, it usually does. That is the worst possible failure
distribution -- it passes where you look and fails where you do not.

The four premature starts, in full:

  api    starts 4.0s before 'postgres' can answer
  api    starts 1.9s before 'qdrant' can answer
  worker starts 3.8s before 'postgres' can answer
  worker starts 0.6s before 'api' can answer

Note the last one. The worker waits for the api, the api waits for postgres,
and none of the waiting means anything, so the error compounds down the chain.

The broken file, which cannot start at all:

  cannot start: dependency cycle: api -> worker -> api
  undefined-dependency  ghost depends on 'nowhere'
  port-conflict         host port 8000 claimed by api and worker

Note that the cycle SUPPRESSES the timing findings. There is no start order, so
"starts too early" is not a meaningful claim, and ready_times would raise.
Reporting a fatal error and a set of warnings derived from an impossible
ordering would be noise.

One implementation note. Under service_started a dependant may begin as soon as
the dependency's CONTAINER exists, which is ready_times[dep] minus that
service's ready_seconds -- not ready_times[dep]. An earlier version used the
latter and the two stacks behaved identically, which quietly destroyed the
entire comparison. test_service_started_does_not_wait_for_the_dependency_to_be_usable
pins it.

run-output.txt

--- depends_on with no condition ---
  start order  postgres -> qdrant -> api -> worker
  usable at    api 1.2s  postgres 4.4s  qdrant 2.3s  worker 1.2s
  [starts-before-ready] api
      starts 4.0s before 'postgres' can answer; depends_on without a condition wai
  [starts-before-ready] api
      starts 1.9s before 'qdrant' can answer; depends_on without a condition waits
  [starts-before-ready] worker
      starts 3.8s before 'postgres' can answer; depends_on without a condition wai
  [starts-before-ready] worker
      starts 0.6s before 'api' can answer; depends_on without a condition waits fo
--- depends_on: service_healthy ---
  start order  postgres -> qdrant -> api -> worker
  usable at    api 5.2s  postgres 4.4s  qdrant 2.3s  worker 5.8s
  no findings
--- a file that cannot start ---
  cannot start: dependency cycle: api -> worker -> api
  [undefined-dependency] ghost
      depends on 'nowhere', which no service defines
  [port-conflict] api, worker
      host port 8000 is claimed by 2 services: api, worker
  [dependency-cycle] api
      cycle: api -> worker -> api; nothing in it can start
--- what the condition changed ---
  api usable at    1.2s  ->  5.2s
  worker usable at 1.2s  ->  5.8s
  premature starts 4  ->  0
  cycles in the broken file: [['api', 'worker']]

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

FAILED tests/test_compose_graph.py::test_a_service_with_no_dependencies_is_ready_after_start_plus_ready
FAILED tests/test_compose_graph.py::test_service_started_does_not_wait_for_the_dependency_to_be_usable
FAILED tests/test_compose_graph.py::test_service_healthy_does_wait - NotImple...
FAILED tests/test_compose_graph.py::test_waiting_properly_is_slower_and_that_is_the_point
FAILED tests/test_compose_graph.py::test_starting_before_a_dependency_can_answer_is_reported
FAILED tests/test_compose_graph.py::test_a_healthy_condition_produces_no_premature_start
FAILED tests/test_compose_graph.py::test_a_dependency_that_is_ready_instantly_is_not_premature
FAILED tests/test_compose_graph.py::test_review_finds_every_class_of_problem
FAILED tests/test_compose_graph.py::test_a_cycle_suppresses_the_timing_findings
FAILED tests/test_compose_graph.py::test_waiting_for_health_on_a_service_with_no_healthcheck_is_reported
FAILED tests/test_compose_graph.py::test_a_correctly_wired_stack_is_clean - N...
26 failed in 0.08s

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

..........................                                               [100%]
26 passed in 0.01s

test-run.txt

..........................                                               [100%]
26 passed in 0.01s

Source files

examples/compose_demo.py (3099 bytes)
#!/usr/bin/env python3
"""One AI stack, wired two ways."""

from __future__ import annotations

import os
import sys

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

from compose_graph import (
    Condition,
    Dependency,
    Service,
    find_cycles,
    ready_times,
    review,
    startup_order,
)


def stack(condition: Condition, healthchecks: bool) -> dict[str, Service]:
    """api -> (postgres, qdrant), worker -> (postgres, api)."""
    return {
        "postgres": Service(
            "postgres", "postgres:16", ports=[(5432, 5432)],
            healthcheck=healthchecks, start_seconds=0.4, ready_seconds=4.0,
        ),
        "qdrant": Service(
            "qdrant", "qdrant/qdrant:v1.12", ports=[(6333, 6333)],
            healthcheck=healthchecks, start_seconds=0.3, ready_seconds=2.0,
        ),
        "api": Service(
            "api", "app/api:1.4", ports=[(8000, 8000)],
            depends_on=[Dependency("postgres", condition), Dependency("qdrant", condition)],
            healthcheck=healthchecks, start_seconds=0.2, ready_seconds=0.6,
        ),
        "worker": Service(
            "worker", "app/worker:1.4",
            depends_on=[Dependency("postgres", condition), Dependency("api", condition)],
            start_seconds=0.2, ready_seconds=0.4,
        ),
    }


NAIVE = stack(Condition.STARTED, healthchecks=False)
CORRECT = stack(Condition.HEALTHY, healthchecks=True)

# A file nobody can start, plus a host port claimed twice.
BROKEN = {
    "api": Service("api", "app/api:1.4", ports=[(8000, 8000)],
                   depends_on=[Dependency("worker")]),
    "worker": Service("worker", "app/worker:1.4", ports=[(8000, 8080)],
                      depends_on=[Dependency("api")]),
    "ghost": Service("ghost", "app/ghost:1.0", depends_on=[Dependency("nowhere")]),
}


def show(name: str, services: dict[str, Service]) -> None:
    print(f"--- {name} ---")
    try:
        print(f"  start order  {' -> '.join(startup_order(services))}")
        times = ready_times(services)
        print("  usable at    " + "  ".join(f"{k} {v}s" for k, v in sorted(times.items())))
    except ValueError as exc:
        print(f"  cannot start: {exc}")
    found = review(services)
    for f in found:
        print(f"  [{f.rule}] {f.service}")
        print(f"      {f.message[:76]}")
    if not found:
        print("  no findings")


def main() -> int:
    show("depends_on with no condition", NAIVE)
    show("depends_on: service_healthy", CORRECT)
    show("a file that cannot start", BROKEN)

    n, c = ready_times(NAIVE), ready_times(CORRECT)
    print("--- what the condition changed ---")
    print(f"  api usable at    {n['api']}s  ->  {c['api']}s")
    print(f"  worker usable at {n['worker']}s  ->  {c['worker']}s")
    print(f"  premature starts {sum(1 for f in review(NAIVE) if f.rule == 'starts-before-ready')}"
          f"  ->  {sum(1 for f in review(CORRECT) if f.rule == 'starts-before-ready')}")
    print(f"  cycles in the broken file: {find_cycles(BROKEN)}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
examples/compose_graph.py (8008 bytes)
"""Work out what a Compose file will actually do before you run it.

Offline and standard-library only. Docker Compose is not required: the input is
a parsed service graph, and the questions worth asking are graph questions.

A multi-service application is a dependency graph, and three things go wrong in
it that are all visible before anything starts:

  cycles          a depends on b depends on a -- nothing can ever start
  port clashes    two services claiming the same host port
  false ordering  depends_on waits for the container, NOT for the service

The third is the one that bites. `depends_on` means "start this container
first". It does not mean "wait until the database can accept a connection", so
an API that starts in 200ms against a Postgres that takes 4 seconds to accept
connections will crash on its first query -- reliably in CI, intermittently on
a laptop, which is the worst possible failure distribution.

The fix is a condition: service_healthy plus a healthcheck. This module shows
the difference the condition makes to the order things are actually usable.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum


class Condition(str, Enum):
    """What `depends_on` actually waits for."""

    STARTED = "service_started"    # the container exists. Says nothing about readiness.
    HEALTHY = "service_healthy"    # the healthcheck passed.
    COMPLETED = "service_completed_successfully"  # a one-shot job finished.


@dataclass(frozen=True)
class Dependency:
    on: str
    condition: Condition = Condition.STARTED


@dataclass
class Service:
    name: str
    image: str
    ports: list[tuple[int, int]] = field(default_factory=list)  # (host, container)
    depends_on: list[Dependency] = field(default_factory=list)
    healthcheck: bool = False
    start_seconds: float = 0.5      # until the container is running
    ready_seconds: float = 0.5      # until it can actually serve

    @property
    def host_ports(self) -> list[int]:
        return [h for h, _ in self.ports]


@dataclass(frozen=True)
class Finding:
    rule: str
    service: str
    message: str


def find_cycles(services: dict[str, Service]) -> list[list[str]]:
    """Every dependency cycle, each reported once starting from its smallest name.

    A cycle means nothing in it can ever start, so this is a hard error rather
    than a warning.
    """
    seen: set[tuple[str, ...]] = set()
    out: list[list[str]] = []

    def walk(node: str, path: list[str]) -> None:
        if node in path:
            cycle = path[path.index(node):]
            rotated = min(
                (tuple(cycle[i:] + cycle[:i]) for i in range(len(cycle))),
                default=(),
            )
            if rotated and rotated not in seen:
                seen.add(rotated)
                out.append(list(rotated))
            return
        svc = services.get(node)
        if not svc:
            return
        for dep in svc.depends_on:
            walk(dep.on, path + [node])

    for name in sorted(services):
        walk(name, [])
    return out


def missing_dependencies(services: dict[str, Service]) -> list[Finding]:
    """Dependencies naming a service the file does not define."""
    out: list[Finding] = []
    for name in sorted(services):
        for dep in services[name].depends_on:
            if dep.on not in services:
                out.append(
                    Finding("undefined-dependency", name,
                            f"depends on '{dep.on}', which no service defines")
                )
    return out


def port_conflicts(services: dict[str, Service]) -> list[Finding]:
    """Two services publishing the same host port.

    Only the HOST side conflicts. Two containers may both listen on 8080
    internally -- they are on different network namespaces.
    """
    claims: dict[int, list[str]] = {}
    for name in sorted(services):
        for port in services[name].host_ports:
            claims.setdefault(port, []).append(name)
    return [
        Finding("port-conflict", ", ".join(names),
                f"host port {port} is claimed by {len(names)} services: {', '.join(names)}")
        for port, names in sorted(claims.items())
        if len(names) > 1
    ]


def startup_order(services: dict[str, Service]) -> list[str]:
    """A topological order, ties broken by name so the result is stable.

    Raises if the graph has a cycle, because there is no valid order.
    """
    cycles = find_cycles(services)
    if cycles:
        raise ValueError(f"dependency cycle: {' -> '.join(cycles[0] + [cycles[0][0]])}")
    order: list[str] = []
    placed: set[str] = set()
    while len(order) < len(services):
        ready = sorted(
            n for n in services
            if n not in placed
            and all(d.on in placed or d.on not in services for d in services[n].depends_on)
        )
        if not ready:
            raise ValueError("no service can start; check the dependency graph")
        for n in ready:
            order.append(n)
            placed.add(n)
    return order


def ready_times(services: dict[str, Service]) -> dict[str, float]:
    """When each service can actually serve, honouring the depends_on condition.

    This is the whole point. With service_started a dependant waits only for the
    container; with service_healthy it waits until the dependency is usable.
    """
    times: dict[str, float] = {}
    for name in startup_order(services):
        svc = services[name]
        begins = 0.0
        for dep in svc.depends_on:
            if dep.on not in services:
                continue
            target = services[dep.on]
            if dep.condition is Condition.STARTED:
                begins = max(begins, times[dep.on] - target.ready_seconds)
            else:
                begins = max(begins, times[dep.on])
        times[name] = round(begins + svc.start_seconds + svc.ready_seconds, 2)
    return times


def premature_starts(services: dict[str, Service]) -> list[Finding]:
    """Services that will begin work before a dependency can answer them."""
    times = ready_times(services)
    out: list[Finding] = []
    for name in sorted(services):
        svc = services[name]
        began = times[name] - svc.ready_seconds - svc.start_seconds
        for dep in svc.depends_on:
            target = services.get(dep.on)
            if not target or dep.condition is not Condition.STARTED:
                continue
            if times[dep.on] > began + 1e-9:
                gap = round(times[dep.on] - began, 2)
                out.append(
                    Finding(
                        "starts-before-ready", name,
                        f"starts {gap}s before '{dep.on}' can answer; depends_on without a "
                        "condition waits for the container, not the service",
                    )
                )
    return out


def review(services: dict[str, Service]) -> list[Finding]:
    """Everything wrong with this graph that is visible without running it."""
    out = missing_dependencies(services) + port_conflicts(services)
    for cycle in find_cycles(services):
        out.append(
            Finding("dependency-cycle", cycle[0],
                    f"cycle: {' -> '.join(cycle + [cycle[0]])}; nothing in it can start")
        )
    if not any(f.rule == "dependency-cycle" for f in out):
        out.extend(premature_starts(services))
    for name in sorted(services):
        svc = services[name]
        depended_on = any(
            d.on == name and d.condition is Condition.HEALTHY
            for other in services.values() for d in other.depends_on
        )
        if depended_on and not svc.healthcheck:
            out.append(
                Finding("healthy-without-healthcheck", name,
                        "another service waits for service_healthy but this one defines no "
                        "healthcheck, so the condition can never be satisfied")
            )
    return out
metadata.yml (974 bytes)
lesson_id: D332
day: 332
kind: python
languages:
  - python
setup_commands:
  - pip install -r requirements/requirements.txt
run_commands:
  - python3 examples/compose_demo.py
test_commands:
  - bash tests/run_tests.sh
cleanup_commands:
  - rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3 examples/compose_demo.py
  reproduced the documented figures exactly (api usable at 1.2s under depends_on with no condition
  against 5.2s under service_healthy, four premature starts falling to zero, and the broken file
  reporting the cycle api -> worker -> api); bash tests/run_tests.sh -> 26 passed in 0.01s. The
  untouched starter fails all 26. requires_network is false and Docker Compose is NOT required: the
  questions worth asking about a Compose file are graph questions, answerable before anything runs.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/compose_graph.py (4621 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.

Work out what a Compose file will actually do before you run it.

Offline and standard-library only. Docker Compose is not required: the input is
a parsed service graph, and the questions worth asking are graph questions.

A multi-service application is a dependency graph, and three things go wrong in
it that are all visible before anything starts:

  cycles          a depends on b depends on a -- nothing can ever start
  port clashes    two services claiming the same host port
  false ordering  depends_on waits for the container, NOT for the service

The third is the one that bites. `depends_on` means "start this container
first". It does not mean "wait until the database can accept a connection", so
an API that starts in 200ms against a Postgres that takes 4 seconds to accept
connections will crash on its first query -- reliably in CI, intermittently on
a laptop, which is the worst possible failure distribution.

The fix is a condition: service_healthy plus a healthcheck. This module shows
the difference the condition makes to the order things are actually usable.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum

class Condition(str, Enum):
    """What `depends_on` actually waits for."""
    STARTED = 'service_started'
    HEALTHY = 'service_healthy'
    COMPLETED = 'service_completed_successfully'

@dataclass(frozen=True)
class Dependency:
    on: str
    condition: Condition = Condition.STARTED

@dataclass
class Service:
    name: str
    image: str
    ports: list[tuple[int, int]] = field(default_factory=list)
    depends_on: list[Dependency] = field(default_factory=list)
    healthcheck: bool = False
    start_seconds: float = 0.5
    ready_seconds: float = 0.5

    @property
    def host_ports(self) -> list[int]:
        raise NotImplementedError('TASK 1: implement host_ports.')

@dataclass(frozen=True)
class Finding:
    rule: str
    service: str
    message: str

def find_cycles(services: dict[str, Service]) -> list[list[str]]:
    """Every dependency cycle, each reported once starting from its smallest name.

    A cycle means nothing in it can ever start, so this is a hard error rather
    than a warning.
    """
    raise NotImplementedError('TASK 3: implement find_cycles. Every dependency cycle, each reported once starting from its smallest name.')

def missing_dependencies(services: dict[str, Service]) -> list[Finding]:
    """Dependencies naming a service the file does not define."""
    raise NotImplementedError('TASK 4: implement missing_dependencies. Dependencies naming a service the file does not define.')

def port_conflicts(services: dict[str, Service]) -> list[Finding]:
    """Two services publishing the same host port.

    Only the HOST side conflicts. Two containers may both listen on 8080
    internally -- they are on different network namespaces.
    """
    raise NotImplementedError('TASK 5: implement port_conflicts. Two services publishing the same host port.')

def startup_order(services: dict[str, Service]) -> list[str]:
    """A topological order, ties broken by name so the result is stable.

    Raises if the graph has a cycle, because there is no valid order.
    """
    raise NotImplementedError('TASK 6: implement startup_order. A topological order, ties broken by name so the result is stable.')

def ready_times(services: dict[str, Service]) -> dict[str, float]:
    """When each service can actually serve, honouring the depends_on condition.

    This is the whole point. With service_started a dependant waits only for the
    container; with service_healthy it waits until the dependency is usable.
    """
    raise NotImplementedError('TASK 7: implement ready_times. When each service can actually serve, honouring the depends_on condition.')

def premature_starts(services: dict[str, Service]) -> list[Finding]:
    """Services that will begin work before a dependency can answer them."""
    raise NotImplementedError('TASK 8: implement premature_starts. Services that will begin work before a dependency can answer them.')

def review(services: dict[str, Service]) -> list[Finding]:
    """Everything wrong with this graph that is visible without running it."""
    raise NotImplementedError('TASK 9: implement review. Everything wrong with this graph that is visible without running it.')
tests/run_tests.sh (211 bytes)
#!/usr/bin/env bash
# Run the day 332 compose-graph test suite.
# Offline, standard library plus pytest. Docker Compose is not required.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_compose_graph.py (6979 bytes)
"""Grouped by what the graph check decides.

Run with: bash tests/run_tests.sh
"""

from __future__ import annotations

import os
import sys

import pytest

HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "examples"))

from compose_graph import (  # noqa: E402
    Condition,
    Dependency,
    Service,
    find_cycles,
    missing_dependencies,
    port_conflicts,
    premature_starts,
    ready_times,
    review,
    startup_order,
)


def svc(name, deps=(), ports=(), healthcheck=False, start=0.5, ready=0.5):
    return Service(
        name, f"img/{name}:1",
        ports=list(ports),
        depends_on=[d if isinstance(d, Dependency) else Dependency(d) for d in deps],
        healthcheck=healthcheck, start_seconds=start, ready_seconds=ready,
    )


def graph(*services):
    return {s.name: s for s in services}


# ------------------------------------------------------------------- cycles


def test_a_simple_cycle_is_found():
    g = graph(svc("a", ["b"]), svc("b", ["a"]))
    assert find_cycles(g) == [["a", "b"]]


def test_a_longer_cycle_is_found():
    g = graph(svc("a", ["b"]), svc("b", ["c"]), svc("c", ["a"]))
    assert find_cycles(g) == [["a", "b", "c"]]


def test_a_cycle_is_reported_once_not_once_per_member():
    g = graph(svc("a", ["b"]), svc("b", ["c"]), svc("c", ["a"]))
    assert len(find_cycles(g)) == 1


def test_a_self_dependency_is_a_cycle():
    assert find_cycles(graph(svc("a", ["a"]))) == [["a"]]


def test_an_acyclic_graph_has_no_cycles():
    g = graph(svc("db"), svc("api", ["db"]), svc("worker", ["api", "db"]))
    assert find_cycles(g) == []


def test_a_diamond_is_not_a_cycle():
    g = graph(svc("db"), svc("a", ["db"]), svc("b", ["db"]), svc("top", ["a", "b"]))
    assert find_cycles(g) == []


# ------------------------------------------------------------ undefined deps


def test_a_dependency_on_an_undefined_service_is_reported():
    f = missing_dependencies(graph(svc("api", ["nowhere"])))
    assert len(f) == 1 and f[0].rule == "undefined-dependency"


def test_defined_dependencies_are_not_reported():
    assert missing_dependencies(graph(svc("db"), svc("api", ["db"]))) == []


# ------------------------------------------------------------------- ports


def test_two_services_claiming_one_host_port_conflict():
    g = graph(svc("a", ports=[(8000, 8000)]), svc("b", ports=[(8000, 8080)]))
    f = port_conflicts(g)
    assert len(f) == 1 and "8000" in f[0].message


def test_the_same_container_port_on_different_host_ports_is_fine():
    # Different network namespaces: only the host side can clash.
    g = graph(svc("a", ports=[(8000, 8080)]), svc("b", ports=[(8001, 8080)]))
    assert port_conflicts(g) == []


def test_services_publishing_nothing_never_conflict():
    assert port_conflicts(graph(svc("a"), svc("b"))) == []


# ------------------------------------------------------------------- order


def test_dependencies_start_before_their_dependants():
    g = graph(svc("db"), svc("api", ["db"]), svc("worker", ["api"]))
    order = startup_order(g)
    assert order.index("db") < order.index("api") < order.index("worker")


def test_the_order_is_stable_for_independent_services():
    g = graph(svc("zebra"), svc("alpha"), svc("middle"))
    assert startup_order(g) == ["alpha", "middle", "zebra"]


def test_a_cycle_has_no_valid_order():
    with pytest.raises(ValueError, match="cycle"):
        startup_order(graph(svc("a", ["b"]), svc("b", ["a"])))


def test_a_dependency_on_an_undefined_service_does_not_block_startup():
    # The file is wrong, but the service is not waiting on anything real.
    assert startup_order(graph(svc("api", ["nowhere"]))) == ["api"]


# -------------------------------------------------------------- readiness


def test_a_service_with_no_dependencies_is_ready_after_start_plus_ready():
    g = graph(svc("db", start=0.4, ready=4.0))
    assert ready_times(g)["db"] == 4.4


def test_service_started_does_not_wait_for_the_dependency_to_be_usable():
    # THE finding: the dependant is "up" while the dependency still cannot answer.
    g = graph(svc("db", start=0.4, ready=4.0), svc("api", ["db"], start=0.2, ready=0.6))
    t = ready_times(g)
    assert t["api"] < t["db"]


def test_service_healthy_does_wait():
    g = graph(
        svc("db", start=0.4, ready=4.0, healthcheck=True),
        svc("api", [Dependency("db", Condition.HEALTHY)], start=0.2, ready=0.6),
    )
    t = ready_times(g)
    assert t["api"] > t["db"]
    assert t["api"] == round(t["db"] + 0.2 + 0.6, 2)


def test_waiting_properly_is_slower_and_that_is_the_point():
    fast = ready_times(graph(svc("db", start=0.4, ready=4.0), svc("api", ["db"], start=0.2, ready=0.6)))
    slow = ready_times(graph(
        svc("db", start=0.4, ready=4.0, healthcheck=True),
        svc("api", [Dependency("db", Condition.HEALTHY)], start=0.2, ready=0.6),
    ))
    assert slow["api"] > fast["api"]


# -------------------------------------------------------- premature starts


def test_starting_before_a_dependency_can_answer_is_reported():
    g = graph(svc("db", start=0.4, ready=4.0), svc("api", ["db"], start=0.2, ready=0.6))
    f = premature_starts(g)
    assert len(f) == 1 and f[0].rule == "starts-before-ready"
    assert "4.0s" in f[0].message


def test_a_healthy_condition_produces_no_premature_start():
    g = graph(
        svc("db", start=0.4, ready=4.0, healthcheck=True),
        svc("api", [Dependency("db", Condition.HEALTHY)], start=0.2, ready=0.6),
    )
    assert premature_starts(g) == []


def test_a_dependency_that_is_ready_instantly_is_not_premature():
    g = graph(svc("db", start=0.1, ready=0.0), svc("api", ["db"], start=0.2, ready=0.6))
    assert premature_starts(g) == []


# ------------------------------------------------------------------ review


def test_review_finds_every_class_of_problem():
    g = graph(
        svc("a", ["b"], ports=[(80, 80)]),
        svc("b", ["a"], ports=[(80, 8080)]),
        svc("ghost", ["nowhere"]),
    )
    rules = {f.rule for f in review(g)}
    assert {"dependency-cycle", "port-conflict", "undefined-dependency"} <= rules


def test_a_cycle_suppresses_the_timing_findings():
    # There is no start order, so "starts too early" is not a meaningful claim.
    g = graph(svc("a", ["b"]), svc("b", ["a"]))
    assert not any(f.rule == "starts-before-ready" for f in review(g))


def test_waiting_for_health_on_a_service_with_no_healthcheck_is_reported():
    g = graph(
        svc("db", healthcheck=False),
        svc("api", [Dependency("db", Condition.HEALTHY)]),
    )
    assert "healthy-without-healthcheck" in {f.rule for f in review(g)}


def test_a_correctly_wired_stack_is_clean():
    g = graph(
        svc("db", ports=[(5432, 5432)], healthcheck=True, start=0.4, ready=4.0),
        svc("api", [Dependency("db", Condition.HEALTHY)], ports=[(8000, 8000)],
            healthcheck=True, start=0.2, ready=0.6),
    )
    assert review(g) == []

Troubleshooting

Troubleshooting — Day 332

NotImplementedError on every test

Expected. The starter stubs nine functions — see expected-output/starter-run.txt.

Start with find_cycles; startup_order depends on it and everything else depends on that.

A cycle is reported three times for a three-service loop

Walking from each service finds the same cycle rotated. Normalise before recording: rotate the cycle so it begins at its smallest member, and keep a set of what you have already emitted.

test_a_self_dependency_is_a_cycle fails

a depending on a is a cycle of length one. If your check only looks for a repeat after at least one step, it misses this — and Compose will refuse to start such a file, so it is worth catching.

test_the_same_container_port_on_different_host_ports_is_fine fails

You are comparing container ports. Two containers may both listen on 8080; they are in different network namespaces. Only the host side can conflict, which is the side ports: binds.

startup_order loops forever

A cycle. Check for one first and raise, rather than spinning waiting for a service that can never become placeable.

test_a_dependency_on_an_undefined_service_does_not_block_startup fails

A dependency naming a service the file does not define is a real problem, and it is reported separately. It must not stop the ordering — the service is not waiting on anything that exists.

test_service_started_does_not_wait_for_the_dependency_to_be_usable fails

This is the finding the lab exists for, so get it right. Under service_started, a dependant may begin as soon as its dependency's container exists — which is ready_times[dep] - dep.ready_seconds, not ready_times[dep]. If you use the latter, the naive and correct stacks behave identically and the whole comparison collapses.

The correct stack is slower and that looks like a failure

It is slower, and it is correct. Under service_healthy the api is usable at 5.2s instead of 1.2s — because at 1.2s it was not usable at all, it was merely running. test_waiting_properly_is_slower_and_that_is_the_point pins this deliberately.

test_a_cycle_suppresses_the_timing_findings fails

review must not run the timing checks when a cycle exists. There is no start order, so "starts too early" is not a meaningful statement — and ready_times will raise.

healthy-without-healthcheck never fires

The check is about the target: some other service waits for service_healthy on it, and it defines no healthcheck. Scan every service's dependencies looking for that condition, then check the named target rather than the waiter.

Security notes

Security notes — Day 332

What this lab does and does not touch

  • No Docker or Compose required and neither is invoked. The input is a parsed service graph. Nothing is started, pulled or networked.
  • No network, no credentials, no registry login.
  • No writes outside this directory. Only __pycache__/, .pytest_cache/ and, if you create it, .venv/.
  • No sudo, no global installs.

Publishing a port is the decision people make by accident

ports: ["5432:5432"] on a database does not "expose it to the API" — the services already reach each other on the Compose network by name. What it does is bind the port on the host, which on a cloud VM with a permissive security group means the internet.

The distinction is worth being precise about, because the wrong mental model produces the wrong file:

  • expose and plain service-to-service traffic need no host binding at all. api reaches postgres:5432 because they share a network.
  • ports publishes to the host. Use it for the one or two services that genuinely take outside traffic.
  • 127.0.0.1:5432:5432 binds to loopback only, which is what you usually want when you need a port for local debugging.

The lab's port-conflict check only looks at the host side, and that is the same reason: the host side is the side with consequences.

Dependency conditions are an availability control

A service that starts before its database can answer does not fail safely. It typically retries, logs an error, and — if the retry has no backoff — hammers the dependency exactly while it is trying to come up. A slow start becomes a failed start.

condition: service_healthy plus a real healthcheck is the fix, and it is worth treating as a reliability control rather than a nicety. Day 342 covers what happens when this is missing during an incident, which is when the ordering matters most and is least likely to be tested.

Secrets do not belong in the file

environment: values live in the Compose file, which lives in Git. Use env_file: for local development with the file gitignored, or Docker secrets, or inject from the orchestrator in production. A POSTGRES_PASSWORD committed to a repository is the most common finding in any first review of a Compose stack.

What a healthcheck should actually check

A healthcheck that returns 200 whenever the process is alive tells you the process is alive, which the container runtime already knew. A useful one verifies the thing a dependant is about to rely on — that the database accepts a connection, that the model is loaded, that the vector index is open.

An unhealthy service that reports healthy is worse than no healthcheck, because now something is waiting on a signal that means nothing.

Reversing everything this lab did

rm -rf __pycache__ .pytest_cache .venv

Nothing else was created, and nothing outside this directory was modified.