Deployment, MLOps, and Security › Containers and Cloud › Day 335

Hands-on lab — Day 335: CI/CD with GitHub Actions

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/pipeline_demo.py

Test

bash tests/run_tests.sh

File tree

examples/pipeline_demo.py
examples/pipeline.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/pipeline.py
tests/run_tests.sh
tests/test_pipeline.py
troubleshooting.md

Lab README

Day 335 Lab: CI/CD with GitHub Actions

Lesson

  • Lesson title: CI/CD with GitHub Actions
  • Day number: 335 of 365
  • Lesson article: https://ai-roadmap-365.github.io/day-335-ci-cd-with-github-actions
  • Lab files: everything you need is in this directory — follow “How to run” below.
  • Browse the course locally: from the repository root, this lab also appears in the course website at /labs/day-335-ci-cd-with-github-actions when the site is running.

Purpose

Reason about a CI pipeline before you push and wait eleven minutes to find out.

A workflow is a job graph, and three questions about it are answerable from the graph alone: how long it actually takes, which jobs can reach a secret, and whether a gate is protecting anything.

You will find that deleting the two jobs that have never caught a bug makes the pipeline zero seconds faster, and that one cache makes it 3.7 minutes faster — because those are different problems.

Learning objectives

  • Compute finish times across a dependency graph where independent jobs run in parallel.
  • Distinguish wall-clock time (the critical path) from runner minutes (the bill).
  • Explain why pull_request_target with secrets is dangerous and pull_request is not.
  • Identify gates that have never fired and caches that are missing.
  • Show what removing a job saves — and why an off-path job saves no time at all.

Prerequisites

  • Day 334, "Cloud Options and Free Tiers".
  • Comfortable with Python dataclasses, enums and basic graph traversal.

Supported operating systems

macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.

Hardware requirements

Any machine running Python 3.10 or newer. The suite completes in hundredths of a second.

Required software

  • Python 3.10 or newer.
  • pytest 7 or newer.

No GitHub account and no runner are used. The shape of a pipeline is the part you can get wrong for months without noticing, and it is checkable offline.

Free and open-source options

Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. GitHub Actions is free for public repositories; act runs workflows locally in containers; GitLab CI, Woodpecker, Drone and Forgejo Actions are open-source alternatives with the same job-graph model.

Installation

python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt

Removing .venv fully reverses the installation.

File structure

starter/pipeline.py        your work: nine stubbed tasks
examples/pipeline.py       reference implementation
examples/pipeline_demo.py  one pipeline read three ways
tests/test_pipeline.py     grouped by what the analysis decides
tests/run_tests.sh         suite entry point
expected-output/           real captured output and measured values
requirements/              pinned dependency

How to run

python3 examples/pipeline_demo.py   # time it, review it, price two changes
bash tests/run_tests.sh             # run the suite

To work on the exercise, edit starter/pipeline.py, then copy it over the reference:

cp starter/pipeline.py examples/pipeline.py
bash tests/run_tests.sh

Keep a copy of the reference first if you want to compare.

What the commands do

python3 examples/pipeline_demo.py takes an eight-job pipeline, computes the critical path and the runner minutes, prints when each job finishes, reviews it for never-firing gates and missing caches, prices the removal of each useless job against a cache on the critical path, and then re-triggers the same jobs under pull_request_target to show what that exposes.

bash tests/run_tests.sh runs pytest over twenty-seven tests grouped by what the analysis decides: timing, critical path, graph validity, review, secrets and removal.

Expected output

  wall clock   12.7 min   along checkout -> install -> integration -> deploy
  runner time  22.2 min billed
--- what removing each never-firing gate would buy ---
  licence-scan   NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 2.4
  docs-build     NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 3.1
--- what caching the install would buy instead ---
  wall 12.7 -> 9.0 min   saves 3.7 min per run

Deleting both useless jobs: 0.0 minutes. One cache on the critical path: 3.7 minutes.

Validation steps

  1. bash tests/run_tests.sh reports 27 passed.
  2. Runner minutes must exceed wall clock whenever any work is parallel. If they are equal everywhere, finish_times is accumulating across every job rather than along dependency edges.
  3. Removing an off-critical-path job must save 0.0 of wall clock. Any other answer means the critical path is wrong.
  4. Removing a job must re-point its dependants onto its dependencies, or the saving is overstated.
  5. secrets-exposed-to-forks must fire on pull_request_target and not on pull_request.

Tests

Twenty-seven tests in six groups:

  • timing — a lone job; dependent jobs adding up; independent jobs in parallel; waiting for the slowest dependency; an undefined dependency not blocking.
  • critical path — the longest chain rather than the sum; runner minutes exceeding wall clock; the path being a real chain; an empty workflow.
  • graph — cycles detected, acyclic graphs clean, timing a cycle raising, undefined needs reported.
  • review — never-firing gates; slow jobs without caches; quick jobs left alone; a cycle suppressing everything else.
  • secrets — pull_request_target with secrets reported; pull_request not; pull_request_target without secrets not.
  • removal — jobs dropped, dependants re-pointed, a missing job raising, and the off-path versus on-path saving.

Cleanup

rm -rf __pycache__ .pytest_cache
rm -rf .venv

Both are safe to remove and fully reversible.

Troubleshooting

See troubleshooting.md.

Security notes

See security.md. In short: pull_request_target runs with your secrets against code a fork controls, third-party actions run their code with your token, and a scanner that has never reported anything is a control that does not exist.

Extension exercises

  1. Add a concurrency limit. Real accounts have a finite number of runners. Recompute wall clock with only two available and see how much of the parallelism was theoretical — for most pipelines it is a great deal.
  2. Add a matrix. Model strategy.matrix expanding one job into several, and find where the runner-minute cost of a matrix outweighs the wall-clock saving.
  3. Analyse your own workflow. Read job durations from a recent run and failure rates from how often each job actually went red this quarter. Most teams have never looked at the second number, and it is the one that tells you which gates to delete.
  • Lesson
  • Previous: Day 334 — Cloud Options and Free Tiers
  • Next: Day 336 — A Containerized AI Deployment

Expected output

FIELDS.md

# Reading the pipeline analysis

## The two durations

    wall clock   12.7 min   along checkout -> install -> integration -> deploy
    runner time  22.2 min billed
    the gap       9.5 min of work happening in parallel

| Number | Meaning | Who cares |
| --- | --- | --- |
| wall clock | The critical path — the longest chain of dependent jobs. | The person waiting for the green tick. |
| runner time | The sum of every job's duration. | Whoever pays the bill. |
| the gap | Work happening in parallel. | Nobody, directly — but it explains why the two differ. |

These are different quantities and they are optimised by different actions.
Confusing them is the mistake this lab exists to prevent.

## The finish table

        0.3 min  checkout
        4.8 min  install
        5.6 min  lint
        7.2 min  licence-scan
        7.9 min  docs-build
        8.0 min  unit
       11.3 min  integration
       12.7 min  deploy

When each job finishes, assuming enough runners. Everything from `lint` to
`integration` starts at 4.8 — they all depend on `install` and on nothing else,
so they run at the same time.

`deploy` waits for `unit`, `integration` and `lint`, so it starts when the last
of those finishes at 11.3, not when the first does.

## The findings

| Rule | Meaning |
| --- | --- |
| `dependency-cycle` | Fatal. No job in the cycle can run, and it suppresses every other finding. |
| `undefined-need` | A `needs:` naming a job the workflow does not define. |
| `secrets-exposed-to-forks` | A secret-using job under `pull_request_target`. See `security.md`. |
| `gate-never-fires` | A job that has never failed. It costs time and there is no evidence it catches anything. |
| `no-cache` | A job of three minutes or more with nothing cached. |

## The result worth reading twice

    licence-scan   NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 2.4
    docs-build     NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 3.1

    install 4.5 -> 0.8 min (a dependency cache)
    wall 12.7 -> 9.0 min   saves 3.7 min per run

**Deleting both useless jobs makes the pipeline zero seconds faster.** They are
not on the critical path, so nothing was waiting for them. It saves 5.5 runner
minutes, which is money, and it is not the thing anyone was complaining about.

One cache on `install` — which *is* on the critical path — saves 3.7 minutes per
run, every run, for everyone.

The general rule: **to make a pipeline faster, shorten the critical path. To
make it cheaper, remove work anywhere.** Those are different jobs, and the
instinct to delete the stage that looks pointless usually addresses the second
while the complaint was about the first.

## Removal re-points dependants

`without` does not merely delete a job. Anything that needed the removed job
inherits that job's own dependencies, because that is what happens when you
delete a stage from a real workflow. Without this, dependants become root jobs,
start immediately, and the wall-clock saving is overstated.

examples-run.txt

--- the pipeline as it stands ---
  wall clock   12.7 min   along checkout -> install -> integration -> deploy
  runner time  22.2 min billed
  the gap      9.5 min of work happening in parallel
--- when each job finishes ---
      0.3 min  checkout
      4.8 min  install
      5.6 min  lint
      7.2 min  licence-scan
      7.9 min  docs-build
      8.0 min  unit
     11.3 min  integration
     12.7 min  deploy
--- findings ---
  [gate-never-fires] docs-build
      has never failed; it costs 3.1 min per run and there is no evidence it catch
  [gate-never-fires] licence-scan
      has never failed; it costs 2.4 min per run and there is no evidence it catch
  [no-cache] docs-build
      takes 3.1 min with nothing cached; a dependency cache is usually the single 
  [no-cache] install
      takes 4.5 min with nothing cached; a dependency cache is usually the single 
--- what removing each never-firing gate would buy ---
  licence-scan   NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 2.4
  docs-build     NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 3.1
--- what caching the install would buy instead ---
  install 4.5 -> 0.8 min (a dependency cache)
  wall 12.7 -> 9.0 min   saves 3.7 min per run
  new critical path: checkout -> install -> integration -> deploy
  deleting BOTH never-firing gates saved 0.0 min of wall clock;
  one cache on the critical path saves 3.7.
--- the same jobs, triggered unsafely ---
  [secrets-exposed-to-forks] deploy
      pull_request_target runs with repository secrets against code the fork contr
  findings of this kind: 1

measured-values.txt

Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.

  python3 examples/pipeline_demo.py -> 35 lines, exit 0
  bash tests/run_tests.sh           -> 27 passed in 0.01s
  the untouched starter             -> 27 failed in 0.08s

One CI pipeline of eight jobs, of the kind that grows a stage at a time.

  wall clock   12.7 min   checkout -> install -> integration -> deploy
  runner time  22.2 min billed
  the gap       9.5 min of parallel work

Two numbers, optimised by different actions. The person waiting cares about the
first; the person paying cares about the second.

The review finds four things:

  gate-never-fires  docs-build     3.1 min per run, never failed
  gate-never-fires  licence-scan   2.4 min per run, never failed
  no-cache          docs-build     3.1 min with nothing cached
  no-cache          install        4.5 min with nothing cached

The obvious move is to delete the two jobs that have never caught anything.
Here is what that buys:

  licence-scan   NOT on the critical path   wall 12.7 -> 12.7   saves 0.0 min
  docs-build     NOT on the critical path   wall 12.7 -> 12.7   saves 0.0 min

Zero. Nothing was waiting for either of them, so removing them cannot make the
pipeline finish sooner. It saves 5.5 runner minutes per run, which is real money
and is not what anybody was complaining about.

What does make it faster:

  install 4.5 -> 0.8 min (a dependency cache)
  wall 12.7 -> 9.0 min   saves 3.7 min per run

One cache, on the one job that everything else waits for.

  deleting BOTH never-firing gates:  0.0 min of wall clock
  one cache on the critical path:    3.7 min

That is the finding. To make a pipeline FASTER, shorten the critical path. To
make it CHEAPER, remove work anywhere. The instinct to delete the stage that
looks pointless addresses the second while the complaint is usually about the
first.

The security finding, on the same jobs with a different trigger:

  secrets-exposed-to-forks  deploy

pull_request_target runs with repository secrets in the context of the base
repository. Combined with a checkout of the fork's code -- which is what people
add next, because otherwise the workflow cannot see the change -- untrusted code
runs with your secrets in the environment. Plain pull_request has a read-only
token and no secrets, and the lab deliberately does not report it, because a
rule that fires on the safe case trains people to ignore the finding that
matters.

One implementation note. `without` re-points dependants: when a job is removed,
anything that needed it inherits its dependencies. An earlier version simply
dropped the edge, which turned dependants into root jobs that start at zero and
overstated the wall-clock saving of every deletion.

run-output.txt

--- the pipeline as it stands ---
  wall clock   12.7 min   along checkout -> install -> integration -> deploy
  runner time  22.2 min billed
  the gap      9.5 min of work happening in parallel
--- when each job finishes ---
      0.3 min  checkout
      4.8 min  install
      5.6 min  lint
      7.2 min  licence-scan
      7.9 min  docs-build
      8.0 min  unit
     11.3 min  integration
     12.7 min  deploy
--- findings ---
  [gate-never-fires] docs-build
      has never failed; it costs 3.1 min per run and there is no evidence it catch
  [gate-never-fires] licence-scan
      has never failed; it costs 2.4 min per run and there is no evidence it catch
  [no-cache] docs-build
      takes 3.1 min with nothing cached; a dependency cache is usually the single 
  [no-cache] install
      takes 4.5 min with nothing cached; a dependency cache is usually the single 
--- what removing each never-firing gate would buy ---
  licence-scan   NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 2.4
  docs-build     NOT on the critical path   wall 12.7 -> 12.7 (saves 0.0)   runner minutes saved 3.1
--- what caching the install would buy instead ---
  install 4.5 -> 0.8 min (a dependency cache)
  wall 12.7 -> 9.0 min   saves 3.7 min per run
  new critical path: checkout -> install -> integration -> deploy
  deleting BOTH never-firing gates saved 0.0 min of wall clock;
  one cache on the critical path saves 3.7.
--- the same jobs, triggered unsafely ---
  [secrets-exposed-to-forks] deploy
      pull_request_target runs with repository secrets against code the fork contr
  findings of this kind: 1

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

FAILED tests/test_pipeline.py::test_a_slow_job_with_a_cache_is_not_reported
FAILED tests/test_pipeline.py::test_a_quick_job_without_a_cache_is_left_alone
FAILED tests/test_pipeline.py::test_a_cycle_suppresses_the_other_findings - N...
FAILED tests/test_pipeline.py::test_pull_request_target_with_secrets_is_reported
FAILED tests/test_pipeline.py::test_the_same_job_on_pull_request_is_not_reported
FAILED tests/test_pipeline.py::test_pull_request_target_without_secrets_is_not_reported
FAILED tests/test_pipeline.py::test_removing_a_job_drops_it - NotImplementedE...
FAILED tests/test_pipeline.py::test_removing_a_job_does_not_orphan_its_dependants
FAILED tests/test_pipeline.py::test_removing_a_job_that_does_not_exist_is_an_error
FAILED tests/test_pipeline.py::test_removing_an_off_path_job_saves_runner_minutes_but_no_wall_clock
FAILED tests/test_pipeline.py::test_removing_an_on_path_job_does_save_wall_clock
27 failed in 0.08s

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

...........................                                              [100%]
27 passed in 0.01s

test-run.txt

...........................                                              [100%]
27 passed in 0.01s

Source files

examples/pipeline_demo.py (3734 bytes)
#!/usr/bin/env python3
"""One CI pipeline, read three ways: how long, how safe, how useful."""

from __future__ import annotations

import os
import sys

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

from pipeline import (
    Job,
    Trigger,
    Workflow,
    critical_path,
    finish_times,
    review,
    savings_from_removing,
    total_minutes,
)

# A pipeline that grew one stage at a time, as they all do.
CURRENT = Workflow(
    "ci",
    triggers=[Trigger.PUSH, Trigger.PULL_REQUEST],
    jobs={
        "checkout":   Job("checkout", [], 0.3, caches=["git"], fails_per_100=0.1),
        "install":    Job("install", ["checkout"], 4.5, fails_per_100=0.4),
        "lint":       Job("lint", ["install"], 0.8, caches=["pip"], fails_per_100=6.0),
        "unit":       Job("unit", ["install"], 3.2, caches=["pip"], fails_per_100=9.0),
        "integration": Job("integration", ["install"], 6.5, caches=["pip"], fails_per_100=2.0),
        "licence-scan": Job("licence-scan", ["install"], 2.4, fails_per_100=0.0),
        "docs-build": Job("docs-build", ["install"], 3.1, fails_per_100=0.0),
        "deploy":     Job("deploy", ["unit", "integration", "lint"], 1.4,
                          uses_secrets=True, fails_per_100=0.6),
    },
)

# The same jobs, but triggered in the way that hands secrets to forks.
UNSAFE = Workflow(
    "ci-unsafe",
    triggers=[Trigger.PUSH, Trigger.PR_TARGET],
    jobs=CURRENT.jobs,
)


def show(label: str, wf: Workflow) -> None:
    path, wall = critical_path(wf)
    print(f"--- {label} ---")
    print(f"  wall clock   {wall} min   along {' -> '.join(path)}")
    print(f"  runner time  {total_minutes(wf)} min billed")
    print(f"  the gap      {round(total_minutes(wf) - wall, 2)} min of work happening in parallel")


def main() -> int:
    show("the pipeline as it stands", CURRENT)

    print("--- when each job finishes ---")
    for name, at in sorted(finish_times(CURRENT).items(), key=lambda kv: (kv[1], kv[0])):
        print(f"    {at:>5.1f} min  {name}")

    print("--- findings ---")
    for f in review(CURRENT):
        print(f"  [{f.rule}] {f.job}")
        print(f"      {f.message[:76]}")

    print("--- what removing each never-firing gate would buy ---")
    for name in ("licence-scan", "docs-build"):
        s = savings_from_removing(CURRENT, name)
        on = "on" if s["on_critical_path"] else "NOT on"
        print(f"  {name:<14} {on} the critical path   "
              f"wall {s['wall_before']} -> {s['wall_after']} (saves {s['wall_saved']})   "
              f"runner minutes saved {s['minutes_saved']}")

    print("--- what caching the install would buy instead ---")
    cached = Workflow(
        CURRENT.name, list(CURRENT.triggers),
        {n: (Job(j.name, list(j.needs), 0.8, j.uses_secrets, ["pip"], j.fails_per_100)
             if n == "install" else j)
         for n, j in CURRENT.jobs.items()},
    )
    _, before = critical_path(CURRENT)
    path_after, after = critical_path(cached)
    print(f"  install 4.5 -> 0.8 min (a dependency cache)")
    print(f"  wall {before} -> {after} min   saves {round(before - after, 2)} min per run")
    print(f"  new critical path: {' -> '.join(path_after)}")
    print(f"  deleting BOTH never-firing gates saved 0.0 min of wall clock;")
    print(f"  one cache on the critical path saves {round(before - after, 2)}.")

    print("--- the same jobs, triggered unsafely ---")
    unsafe = [f for f in review(UNSAFE) if f.rule == "secrets-exposed-to-forks"]
    for f in unsafe:
        print(f"  [{f.rule}] {f.job}")
        print(f"      {f.message[:76]}")
    print(f"  findings of this kind: {len(unsafe)}")
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
examples/pipeline.py (7719 bytes)
"""Reason about a CI pipeline before you push and wait eleven minutes to find out.

Offline and standard-library only. No runner, no GitHub, no network. The input
is a described workflow -- jobs, what each needs, how long it takes, how often
it fails -- and the questions worth asking are answerable from that.

Three of them:

  how long   the critical path, not the sum. Independent jobs run in parallel.
  how safe   which jobs can reach a secret, and whether a fork can trigger them
  how useful a gate that never fails is not a gate

The third is the one that decays quietly. A test job that has not failed in six
months is either protecting nothing or being ignored, and the pipeline gets
slower every quarter as people add stages nobody has ever seen go red.

Nothing here runs your tests. It reasons about the SHAPE of a pipeline, which
is the part you can get wrong for months without noticing.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum


class Trigger(str, Enum):
    PUSH = "push"                          # a branch in your own repository
    PULL_REQUEST = "pull_request"          # read-only token, no secrets by default
    PR_TARGET = "pull_request_target"      # runs with SECRETS against fork code
    SCHEDULE = "schedule"
    MANUAL = "workflow_dispatch"


@dataclass
class Job:
    name: str
    needs: list[str] = field(default_factory=list)
    minutes: float = 1.0
    uses_secrets: bool = False
    caches: list[str] = field(default_factory=list)
    fails_per_100: float = 0.0     # how often this job catches something


@dataclass
class Workflow:
    name: str
    triggers: list[Trigger]
    jobs: dict[str, Job]


def missing_needs(wf: Workflow) -> list[str]:
    """Dependencies naming a job the workflow does not define."""
    return sorted(
        f"{name} needs '{dep}', which no job defines"
        for name, job in wf.jobs.items()
        for dep in job.needs
        if dep not in wf.jobs
    )


def find_cycle(wf: Workflow) -> list[str]:
    """One dependency cycle if any exists, normalised so it is stable."""
    def walk(node: str, path: list[str]) -> list[str] | None:
        if node in path:
            cycle = path[path.index(node):]
            return list(min(tuple(cycle[i:] + cycle[:i]) for i in range(len(cycle))))
        for dep in wf.jobs[node].needs if node in wf.jobs else []:
            found = walk(dep, path + [node])
            if found:
                return found
        return None

    for name in sorted(wf.jobs):
        found = walk(name, [])
        if found:
            return found
    return []


def finish_times(wf: Workflow) -> dict[str, float]:
    """When each job finishes, assuming unlimited parallel runners.

    A job starts as soon as everything it needs has finished, so its finish
    time is the latest of its dependencies plus its own duration.
    """
    if find_cycle(wf):
        raise ValueError(f"dependency cycle: {' -> '.join(find_cycle(wf))}")
    times: dict[str, float] = {}
    remaining = dict(wf.jobs)
    while remaining:
        ready = [n for n, j in remaining.items()
                 if all(d in times or d not in wf.jobs for d in j.needs)]
        if not ready:
            raise ValueError("no job can start; check the dependency graph")
        for name in sorted(ready):
            job = remaining.pop(name)
            start = max((times[d] for d in job.needs if d in times), default=0.0)
            times[name] = round(start + job.minutes, 2)
    return times


def critical_path(wf: Workflow) -> tuple[list[str], float]:
    """The longest chain of dependent jobs, and how long it takes.

    This is what a pipeline actually costs in wall-clock time. Summing the job
    durations overstates it, sometimes by a lot, because independent jobs run
    at the same time.
    """
    times = finish_times(wf)
    if not times:
        return [], 0.0
    end = max(times, key=lambda n: (times[n], n))
    chain = [end]
    while wf.jobs[chain[-1]].needs:
        deps = [d for d in wf.jobs[chain[-1]].needs if d in times]
        if not deps:
            break
        chain.append(max(deps, key=lambda d: (times[d], d)))
    return list(reversed(chain)), round(times[end], 2)


def total_minutes(wf: Workflow) -> float:
    """Runner minutes consumed. What you are billed for, and not what you wait."""
    return round(sum(j.minutes for j in wf.jobs.values()), 2)


@dataclass(frozen=True)
class Finding:
    rule: str
    job: str
    message: str


def review(wf: Workflow) -> list[Finding]:
    """Everything wrong with this pipeline's shape."""
    out: list[Finding] = []

    for msg in missing_needs(wf):
        out.append(Finding("undefined-need", msg.split()[0], msg))

    cycle = find_cycle(wf)
    if cycle:
        out.append(
            Finding("dependency-cycle", cycle[0],
                    f"cycle: {' -> '.join(cycle + [cycle[0]])}; no job in it can run")
        )
        return out

    # pull_request_target runs with repository secrets against code from a fork.
    if Trigger.PR_TARGET in wf.triggers:
        for name, job in sorted(wf.jobs.items()):
            if job.uses_secrets:
                out.append(
                    Finding("secrets-exposed-to-forks", name,
                            "pull_request_target runs with repository secrets against code the "
                            "fork controls; a pull request can exfiltrate them")
                )

    # A gate that has never failed is not protecting anything you can point to.
    for name, job in sorted(wf.jobs.items()):
        if job.fails_per_100 == 0.0 and job.minutes > 0:
            out.append(
                Finding("gate-never-fires", name,
                        f"has never failed; it costs {job.minutes} min per run and there is no "
                        "evidence it catches anything")
            )

    # A slow job with no cache is the cheapest thing to fix.
    for name, job in sorted(wf.jobs.items()):
        if job.minutes >= 3.0 and not job.caches:
            out.append(
                Finding("no-cache", name,
                        f"takes {job.minutes} min with nothing cached; a dependency cache is "
                        "usually the single largest saving available")
            )
    return out


def without(wf: Workflow, job_name: str) -> Workflow:
    """The same workflow with one job removed, and dependants re-pointed.

    Removing a job must not orphan the jobs that needed it -- they inherit its
    dependencies, which is what actually happens when you delete a stage.
    """
    if job_name not in wf.jobs:
        raise KeyError(job_name)
    inherited = wf.jobs[job_name].needs
    jobs = {}
    for name, job in wf.jobs.items():
        if name == job_name:
            continue
        needs = [d for d in job.needs if d != job_name]
        if job_name in job.needs:
            needs += [d for d in inherited if d not in needs]
        jobs[name] = Job(job.name, needs, job.minutes, job.uses_secrets,
                         list(job.caches), job.fails_per_100)
    return Workflow(wf.name, list(wf.triggers), jobs)


def savings_from_removing(wf: Workflow, job_name: str) -> dict[str, float]:
    """What deleting one job buys in wall-clock and in runner minutes."""
    before_path, before_wall = critical_path(wf)
    after = without(wf, job_name)
    _, after_wall = critical_path(after)
    return {
        "wall_before": before_wall,
        "wall_after": after_wall,
        "wall_saved": round(before_wall - after_wall, 2),
        "minutes_saved": round(total_minutes(wf) - total_minutes(after), 2),
        "on_critical_path": job_name in before_path,
    }
metadata.yml (999 bytes)
lesson_id: D335
day: 335
kind: python
languages:
  - python
setup_commands:
  - pip install -r requirements/requirements.txt
run_commands:
  - python3 examples/pipeline_demo.py
test_commands:
  - bash tests/run_tests.sh
cleanup_commands:
  - rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3 examples/pipeline_demo.py
  reproduced the documented figures exactly (12.7 min wall clock against 22.2 runner minutes; deleting
  both never-firing gates saving 0.0 min of wall clock while one cache on the critical path saves 3.7;
  and pull_request_target reporting the deploy job as exposed); bash tests/run_tests.sh -> 27 passed in
  0.01s. The untouched starter fails all 27. requires_network is false and NO GitHub account or runner
  is used: the shape of a pipeline is what you can get wrong for months, and it is checkable offline.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/pipeline.py (4353 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.

Reason about a CI pipeline before you push and wait eleven minutes to find out.

Offline and standard-library only. No runner, no GitHub, no network. The input
is a described workflow -- jobs, what each needs, how long it takes, how often
it fails -- and the questions worth asking are answerable from that.

Three of them:

  how long   the critical path, not the sum. Independent jobs run in parallel.
  how safe   which jobs can reach a secret, and whether a fork can trigger them
  how useful a gate that never fails is not a gate

The third is the one that decays quietly. A test job that has not failed in six
months is either protecting nothing or being ignored, and the pipeline gets
slower every quarter as people add stages nobody has ever seen go red.

Nothing here runs your tests. It reasons about the SHAPE of a pipeline, which
is the part you can get wrong for months without noticing.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum

class Trigger(str, Enum):
    PUSH = 'push'
    PULL_REQUEST = 'pull_request'
    PR_TARGET = 'pull_request_target'
    SCHEDULE = 'schedule'
    MANUAL = 'workflow_dispatch'

@dataclass
class Job:
    name: str
    needs: list[str] = field(default_factory=list)
    minutes: float = 1.0
    uses_secrets: bool = False
    caches: list[str] = field(default_factory=list)
    fails_per_100: float = 0.0

@dataclass
class Workflow:
    name: str
    triggers: list[Trigger]
    jobs: dict[str, Job]

def missing_needs(wf: Workflow) -> list[str]:
    """Dependencies naming a job the workflow does not define."""
    raise NotImplementedError('TASK 1: implement missing_needs. Dependencies naming a job the workflow does not define.')

def find_cycle(wf: Workflow) -> list[str]:
    """One dependency cycle if any exists, normalised so it is stable."""
    raise NotImplementedError('TASK 3: implement find_cycle. One dependency cycle if any exists, normalised so it is stable.')

def finish_times(wf: Workflow) -> dict[str, float]:
    """When each job finishes, assuming unlimited parallel runners.

    A job starts as soon as everything it needs has finished, so its finish
    time is the latest of its dependencies plus its own duration.
    """
    raise NotImplementedError('TASK 4: implement finish_times. When each job finishes, assuming unlimited parallel runners.')

def critical_path(wf: Workflow) -> tuple[list[str], float]:
    """The longest chain of dependent jobs, and how long it takes.

    This is what a pipeline actually costs in wall-clock time. Summing the job
    durations overstates it, sometimes by a lot, because independent jobs run
    at the same time.
    """
    raise NotImplementedError('TASK 5: implement critical_path. The longest chain of dependent jobs, and how long it takes.')

def total_minutes(wf: Workflow) -> float:
    """Runner minutes consumed. What you are billed for, and not what you wait."""
    raise NotImplementedError('TASK 6: implement total_minutes. Runner minutes consumed. What you are billed for, and not what you wait.')

@dataclass(frozen=True)
class Finding:
    rule: str
    job: str
    message: str

def review(wf: Workflow) -> list[Finding]:
    """Everything wrong with this pipeline's shape."""
    raise NotImplementedError("TASK 7: implement review. Everything wrong with this pipeline's shape.")

def without(wf: Workflow, job_name: str) -> Workflow:
    """The same workflow with one job removed, and dependants re-pointed.

    Removing a job must not orphan the jobs that needed it -- they inherit its
    dependencies, which is what actually happens when you delete a stage.
    """
    raise NotImplementedError('TASK 8: implement without. The same workflow with one job removed, and dependants re-pointed.')

def savings_from_removing(wf: Workflow, job_name: str) -> dict[str, float]:
    """What deleting one job buys in wall-clock and in runner minutes."""
    raise NotImplementedError('TASK 9: implement savings_from_removing. What deleting one job buys in wall-clock and in runner minutes.')
tests/run_tests.sh (210 bytes)
#!/usr/bin/env bash
# Run the day 335 CI pipeline test suite.
# Offline, standard library plus pytest. No runner and no GitHub account.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_pipeline.py (6537 bytes)
"""Grouped by what the pipeline analysis decides.

Run with: bash tests/run_tests.sh
"""

from __future__ import annotations

import os
import sys

import pytest

HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "examples"))

from pipeline import (  # noqa: E402
    Job,
    Trigger,
    Workflow,
    critical_path,
    find_cycle,
    finish_times,
    missing_needs,
    review,
    savings_from_removing,
    total_minutes,
    without,
)


def wf(jobs, triggers=(Trigger.PUSH,)):
    return Workflow("ci", list(triggers), {j.name: j for j in jobs})


def job(name, needs=(), minutes=1.0, secrets=False, caches=(), fails=1.0):
    return Job(name, list(needs), minutes, secrets, list(caches), fails)


# ------------------------------------------------------------------ timing


def test_a_single_job_finishes_after_its_own_duration():
    assert finish_times(wf([job("a", minutes=2.5)])) == {"a": 2.5}


def test_dependent_jobs_add_up():
    t = finish_times(wf([job("a", minutes=2.0), job("b", ["a"], minutes=3.0)]))
    assert t["b"] == 5.0


def test_independent_jobs_run_in_parallel_not_in_sequence():
    # The whole reason wall clock and runner minutes differ.
    t = finish_times(wf([job("a", minutes=2.0), job("b", minutes=3.0)]))
    assert t["a"] == 2.0 and t["b"] == 3.0


def test_a_job_waits_for_its_slowest_dependency():
    t = finish_times(wf([
        job("fast", minutes=1.0), job("slow", minutes=5.0),
        job("after", ["fast", "slow"], minutes=1.0),
    ]))
    assert t["after"] == 6.0


def test_a_dependency_on_an_undefined_job_does_not_block_timing():
    t = finish_times(wf([job("a", ["nowhere"], minutes=2.0)]))
    assert t["a"] == 2.0


# ------------------------------------------------------------ critical path


def test_the_critical_path_is_the_longest_chain_not_the_sum():
    w = wf([job("a", minutes=2.0), job("b", ["a"], minutes=3.0), job("side", minutes=10.0)])
    path, wall = critical_path(w)
    assert wall == 10.0 and path == ["side"]
    assert total_minutes(w) == 15.0


def test_runner_minutes_exceed_wall_clock_when_work_is_parallel():
    w = wf([job("a", minutes=4.0), job("b", minutes=4.0), job("c", minutes=4.0)])
    _, wall = critical_path(w)
    assert wall == 4.0
    assert total_minutes(w) == 12.0


def test_the_critical_path_is_a_real_chain_of_dependencies():
    w = wf([job("a", minutes=1.0), job("b", ["a"], minutes=2.0), job("c", ["b"], minutes=3.0)])
    path, wall = critical_path(w)
    assert path == ["a", "b", "c"] and wall == 6.0


def test_an_empty_workflow_has_no_critical_path():
    assert critical_path(wf([])) == ([], 0.0)


# ------------------------------------------------------------------ graph


def test_a_cycle_is_detected():
    assert find_cycle(wf([job("a", ["b"]), job("b", ["a"])])) == ["a", "b"]


def test_an_acyclic_graph_has_no_cycle():
    assert find_cycle(wf([job("a"), job("b", ["a"])])) == []


def test_timing_a_cyclic_workflow_raises():
    with pytest.raises(ValueError, match="cycle"):
        finish_times(wf([job("a", ["b"]), job("b", ["a"])]))


def test_an_undefined_dependency_is_reported():
    assert missing_needs(wf([job("a", ["ghost"])])) == ["a needs 'ghost', which no job defines"]


# ------------------------------------------------------------------ review


def test_a_gate_that_has_never_failed_is_reported():
    w = wf([job("scan", minutes=2.0, fails=0.0)])
    assert "gate-never-fires" in {f.rule for f in review(w)}


def test_a_gate_that_does_fail_is_not_reported():
    w = wf([job("unit", minutes=2.0, caches=["pip"], fails=5.0)])
    assert "gate-never-fires" not in {f.rule for f in review(w)}


def test_a_slow_job_with_no_cache_is_reported():
    assert "no-cache" in {f.rule for f in review(wf([job("install", minutes=4.0)]))}


def test_a_slow_job_with_a_cache_is_not_reported():
    w = wf([job("install", minutes=4.0, caches=["pip"])])
    assert "no-cache" not in {f.rule for f in review(w)}


def test_a_quick_job_without_a_cache_is_left_alone():
    assert "no-cache" not in {f.rule for f in review(wf([job("tiny", minutes=0.5)]))}


def test_a_cycle_suppresses_the_other_findings():
    w = wf([job("a", ["b"], minutes=9.0, fails=0.0), job("b", ["a"], minutes=9.0, fails=0.0)])
    rules = {f.rule for f in review(w)}
    assert rules == {"dependency-cycle"}


# ---------------------------------------------------------------- secrets


def test_pull_request_target_with_secrets_is_reported():
    w = wf([job("deploy", minutes=1.0, secrets=True)],
           triggers=(Trigger.PUSH, Trigger.PR_TARGET))
    assert "secrets-exposed-to-forks" in {f.rule for f in review(w)}


def test_the_same_job_on_pull_request_is_not_reported():
    w = wf([job("deploy", minutes=1.0, secrets=True)],
           triggers=(Trigger.PUSH, Trigger.PULL_REQUEST))
    assert "secrets-exposed-to-forks" not in {f.rule for f in review(w)}


def test_pull_request_target_without_secrets_is_not_reported():
    w = wf([job("build", minutes=1.0, secrets=False)],
           triggers=(Trigger.PR_TARGET,))
    assert "secrets-exposed-to-forks" not in {f.rule for f in review(w)}


# ---------------------------------------------------------------- removal


def test_removing_a_job_drops_it():
    w = without(wf([job("a"), job("b")]), "a")
    assert set(w.jobs) == {"b"}


def test_removing_a_job_does_not_orphan_its_dependants():
    # b needed a, a needed checkout. Remove a and b must still wait for checkout.
    w = without(wf([job("checkout"), job("a", ["checkout"]), job("b", ["a"])]), "a")
    assert w.jobs["b"].needs == ["checkout"]


def test_removing_a_job_that_does_not_exist_is_an_error():
    with pytest.raises(KeyError):
        without(wf([job("a")]), "nope")


def test_removing_an_off_path_job_saves_runner_minutes_but_no_wall_clock():
    # THE finding: deleting a slow job that nothing waits for makes the
    # pipeline cheaper and not one second faster.
    w = wf([
        job("install", minutes=4.0),
        job("long", ["install"], minutes=6.0),
        job("side", ["install"], minutes=3.0, fails=0.0),
    ])
    s = savings_from_removing(w, "side")
    assert s["on_critical_path"] is False
    assert s["wall_saved"] == 0.0
    assert s["minutes_saved"] == 3.0


def test_removing_an_on_path_job_does_save_wall_clock():
    w = wf([job("install", minutes=4.0), job("long", ["install"], minutes=6.0)])
    s = savings_from_removing(w, "long")
    assert s["on_critical_path"] is True
    assert s["wall_saved"] == 6.0

Troubleshooting

Troubleshooting — Day 335

NotImplementedError on every test

Expected. The starter stubs nine functions — see expected-output/starter-run.txt.

Start with finish_times; the critical path, the review and the savings all read from it.

test_independent_jobs_run_in_parallel_not_in_sequence fails

You are accumulating durations across every job rather than only along dependency edges. A job's finish time is max(finish of its needs) + its own duration, and a job with no needs starts at zero regardless of what else exists.

If this is wrong, wall clock equals runner minutes everywhere and the entire lesson disappears.

test_the_critical_path_is_the_longest_chain_not_the_sum fails

The critical path is the chain ending at the latest-finishing job, walked backwards through whichever dependency finished last. It is not the sum of all durations, and it is not necessarily the longest chain by number of jobs.

finish_times loops forever

A cycle. Check for one and raise before the scheduling loop, rather than spinning waiting for a job that can never become ready.

test_a_cycle_suppresses_the_other_findings fails

review must return the cycle finding and stop. With no valid ordering there are no finish times, so every timing-derived finding would either be wrong or raise. A fatal finding should suppress the derived ones.

test_removing_a_job_does_not_orphan_its_dependants fails

This is the subtle one. When you delete job a, anything that needed a must inherit a's own dependencies — otherwise b becomes a root job and starts immediately, which silently understates the wall clock and makes deletion look better than it is.

savings_from_removing reports wall-clock savings for an off-path job

Then your critical path is wrong, because removing a job nothing waits for cannot make the pipeline finish sooner. Getting 0.0 here is the correct and instructive answer.

secrets-exposed-to-forks fires on pull_request

Only pull_request_target hands repository secrets to a workflow running against fork-controlled code. Plain pull_request runs with a read-only token and no secrets, and reporting it produces noise that trains people to ignore the finding that matters.

no-cache fires on everything

The rule is for jobs that are slow — three minutes or more — and have no cache declared. A half-minute job without a cache is not worth a finding, and a rule that fires on every job is a rule nobody reads.

The numbers do not match my pipeline

They will not. The durations and failure rates in the demo are illustrative. Put your own in: you can read job durations from any recent run, and the failure rates from how often each job has actually gone red this quarter — which is a number most teams have never looked at.

Security notes

Security notes — Day 335

What this lab does and does not touch

  • No GitHub account, no runner, no workflow is executed. The input is a described pipeline. Nothing is dispatched and no token is read.
  • No network, no credentials.
  • No writes outside this directory. Only __pycache__/, .pytest_cache/ and, if you create it, .venv/.
  • No sudo, no global installs.

pull_request_target is the one to understand

This is the single most dangerous line in GitHub Actions, and it is dangerous because it looks like a small variation on a safe thing.

  • pull_request checks out the fork's code and runs with a read-only token and no secrets. A hostile pull request can run whatever it likes and gain nothing.
  • pull_request_target runs in the context of the base repository — with your secrets available — and exists so that a workflow can label or comment on a PR from a fork.

The trap is combining it with a checkout of the fork's code. At that point untrusted code executes with your repository secrets in the environment, and exfiltrating them is a one-line change to a test file or a Makefile. This has been used against real projects repeatedly.

The rule that survives: pull_request_target may inspect a pull request; it must never run its code. If you need both, split into two workflows — one that runs untrusted code with no secrets, one that reacts to the result.

The lab reports every secret-using job under a pull_request_target trigger for this reason.

Least privilege for the token

GITHUB_TOKEN defaults to broad permissions in older repositories. Set them explicitly, at the top of the workflow and narrowed per job:

permissions:
  contents: read

Then grant only what a specific job needs. A lint job needs nothing; a job that pushes a release needs contents: write and nothing else.

Third-party actions are supply chain

uses: some-org/some-action@v3 executes somebody else's code with your token. A tag is mutable — the author, or anyone who compromises them, can move v3. Pin to a full commit SHA for anything that touches secrets, and read what you are pinning.

This is Day 347's subject and it belongs here too, because CI is where most organisations run the most untrusted code with the most privilege.

Caches are writable by the jobs that read them

A cache is keyed content that later runs restore and trust. A workflow that can be triggered from a fork and writes to a shared cache can poison what a later privileged run restores. Scope cache keys so untrusted runs cannot write to a key a trusted run reads.

A gate that never fails is a security problem too

The lab reports jobs that have never failed. The framing is usually about time, and there is a security version: a scanner that has never reported anything is either configured wrongly or nobody reads its output. Both mean the control does not exist while appearing on the diagram.

Reversing everything this lab did

rm -rf __pycache__ .pytest_cache .venv

Nothing else was created, and nothing outside this directory was modified.