Deployment, MLOps, and Security › Containers and Cloud › Day 335
Hands-on lab — Day 335: CI/CD with GitHub Actions
- ← Back to the Day 335 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/deployment-mlops-and-security/day-335-ci-cd-with-github-actions/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/pipeline_demo.py Test
bash tests/run_tests.sh File tree
examples/pipeline_demo.py examples/pipeline.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/pipeline.py tests/run_tests.sh tests/test_pipeline.py troubleshooting.md
Lab README
Day 335 Lab: CI/CD with GitHub Actions
Lesson
- Lesson title: CI/CD with GitHub Actions
- Day number: 335 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-335-ci-cd-with-github-actions
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-335-ci-cd-with-github-actionswhen the site is running.
Purpose
Reason about a CI pipeline before you push and wait eleven minutes to find out.
A workflow is a job graph, and three questions about it are answerable from the graph alone: how long it actually takes, which jobs can reach a secret, and whether a gate is protecting anything.
You will find that deleting the two jobs that have never caught a bug makes the pipeline zero seconds faster, and that one cache makes it 3.7 minutes faster — because those are different problems.
Learning objectives
- Compute finish times across a dependency graph where independent jobs run in parallel.
- Distinguish wall-clock time (the critical path) from runner minutes (the bill).
- Explain why
pull_request_targetwith secrets is dangerous andpull_requestis not. - Identify gates that have never fired and caches that are missing.
- Show what removing a job saves — and why an off-path job saves no time at all.
Prerequisites
- Day 334, "Cloud Options and Free Tiers".
- Comfortable with Python dataclasses, enums and basic graph traversal.
Supported operating systems
macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.
Hardware requirements
Any machine running Python 3.10 or newer. The suite completes in hundredths of a second.
Required software
- Python 3.10 or newer.
pytest7 or newer.
No GitHub account and no runner are used. The shape of a pipeline is the part you can get wrong for months without noticing, and it is checkable offline.
Free and open-source options
Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. GitHub Actions is free for public repositories; act runs workflows locally in containers; GitLab CI, Woodpecker, Drone and Forgejo Actions are open-source alternatives with the same job-graph model.
Installation
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
Removing .venv fully reverses the installation.
File structure
starter/pipeline.py your work: nine stubbed tasks
examples/pipeline.py reference implementation
examples/pipeline_demo.py one pipeline read three ways
tests/test_pipeline.py grouped by what the analysis decides
tests/run_tests.sh suite entry point
expected-output/ real captured output and measured values
requirements/ pinned dependency
How to run
python3 examples/pipeline_demo.py # time it, review it, price two changes
bash tests/run_tests.sh # run the suite
To work on the exercise, edit starter/pipeline.py, then copy it over the reference:
cp starter/pipeline.py examples/pipeline.py
bash tests/run_tests.sh
Keep a copy of the reference first if you want to compare.
What the commands do
python3 examples/pipeline_demo.py takes an eight-job pipeline, computes the critical path and the runner minutes, prints when each job finishes, reviews it for never-firing gates and missing caches, prices the removal of each useless job against a cache on the critical path, and then re-triggers the same jobs under pull_request_target to show what that exposes.
bash tests/run_tests.sh runs pytest over twenty-seven tests grouped by what the analysis decides: timing, critical path, graph validity, review, secrets and removal.
Expected output
wall clock 12.7 min along checkout -> install -> integration -> deploy
runner time 22.2 min billed
--- what removing each never-firing gate would buy ---
licence-scan NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 2.4
docs-build NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 3.1
--- what caching the install would buy instead ---
wall 12.7 -> 9.0 min saves 3.7 min per run
Deleting both useless jobs: 0.0 minutes. One cache on the critical path: 3.7 minutes.
Validation steps
bash tests/run_tests.shreports27 passed.- Runner minutes must exceed wall clock whenever any work is parallel. If they are equal everywhere,
finish_timesis accumulating across every job rather than along dependency edges. - Removing an off-critical-path job must save
0.0of wall clock. Any other answer means the critical path is wrong. - Removing a job must re-point its dependants onto its dependencies, or the saving is overstated.
secrets-exposed-to-forksmust fire onpull_request_targetand not onpull_request.
Tests
Twenty-seven tests in six groups:
- timing — a lone job; dependent jobs adding up; independent jobs in parallel; waiting for the slowest dependency; an undefined dependency not blocking.
- critical path — the longest chain rather than the sum; runner minutes exceeding wall clock; the path being a real chain; an empty workflow.
- graph — cycles detected, acyclic graphs clean, timing a cycle raising, undefined needs reported.
- review — never-firing gates; slow jobs without caches; quick jobs left alone; a cycle suppressing everything else.
- secrets —
pull_request_targetwith secrets reported;pull_requestnot;pull_request_targetwithout secrets not. - removal — jobs dropped, dependants re-pointed, a missing job raising, and the off-path versus on-path saving.
Cleanup
rm -rf __pycache__ .pytest_cache
rm -rf .venv
Both are safe to remove and fully reversible.
Troubleshooting
See troubleshooting.md.
Security notes
See security.md. In short: pull_request_target runs with your secrets against code a fork controls, third-party actions run their code with your token, and a scanner that has never reported anything is a control that does not exist.
Extension exercises
- Add a concurrency limit. Real accounts have a finite number of runners. Recompute wall clock with only two available and see how much of the parallelism was theoretical — for most pipelines it is a great deal.
- Add a matrix. Model
strategy.matrixexpanding one job into several, and find where the runner-minute cost of a matrix outweighs the wall-clock saving. - Analyse your own workflow. Read job durations from a recent run and failure rates from how often each job actually went red this quarter. Most teams have never looked at the second number, and it is the one that tells you which gates to delete.
Navigation
- Lesson
- Previous: Day 334 — Cloud Options and Free Tiers
- Next: Day 336 — A Containerized AI Deployment
Expected output
FIELDS.md
# Reading the pipeline analysis
## The two durations
wall clock 12.7 min along checkout -> install -> integration -> deploy
runner time 22.2 min billed
the gap 9.5 min of work happening in parallel
| Number | Meaning | Who cares |
| --- | --- | --- |
| wall clock | The critical path — the longest chain of dependent jobs. | The person waiting for the green tick. |
| runner time | The sum of every job's duration. | Whoever pays the bill. |
| the gap | Work happening in parallel. | Nobody, directly — but it explains why the two differ. |
These are different quantities and they are optimised by different actions.
Confusing them is the mistake this lab exists to prevent.
## The finish table
0.3 min checkout
4.8 min install
5.6 min lint
7.2 min licence-scan
7.9 min docs-build
8.0 min unit
11.3 min integration
12.7 min deploy
When each job finishes, assuming enough runners. Everything from `lint` to
`integration` starts at 4.8 — they all depend on `install` and on nothing else,
so they run at the same time.
`deploy` waits for `unit`, `integration` and `lint`, so it starts when the last
of those finishes at 11.3, not when the first does.
## The findings
| Rule | Meaning |
| --- | --- |
| `dependency-cycle` | Fatal. No job in the cycle can run, and it suppresses every other finding. |
| `undefined-need` | A `needs:` naming a job the workflow does not define. |
| `secrets-exposed-to-forks` | A secret-using job under `pull_request_target`. See `security.md`. |
| `gate-never-fires` | A job that has never failed. It costs time and there is no evidence it catches anything. |
| `no-cache` | A job of three minutes or more with nothing cached. |
## The result worth reading twice
licence-scan NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 2.4
docs-build NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 3.1
install 4.5 -> 0.8 min (a dependency cache)
wall 12.7 -> 9.0 min saves 3.7 min per run
**Deleting both useless jobs makes the pipeline zero seconds faster.** They are
not on the critical path, so nothing was waiting for them. It saves 5.5 runner
minutes, which is money, and it is not the thing anyone was complaining about.
One cache on `install` — which *is* on the critical path — saves 3.7 minutes per
run, every run, for everyone.
The general rule: **to make a pipeline faster, shorten the critical path. To
make it cheaper, remove work anywhere.** Those are different jobs, and the
instinct to delete the stage that looks pointless usually addresses the second
while the complaint was about the first.
## Removal re-points dependants
`without` does not merely delete a job. Anything that needed the removed job
inherits that job's own dependencies, because that is what happens when you
delete a stage from a real workflow. Without this, dependants become root jobs,
start immediately, and the wall-clock saving is overstated.
examples-run.txt
--- the pipeline as it stands ---
wall clock 12.7 min along checkout -> install -> integration -> deploy
runner time 22.2 min billed
the gap 9.5 min of work happening in parallel
--- when each job finishes ---
0.3 min checkout
4.8 min install
5.6 min lint
7.2 min licence-scan
7.9 min docs-build
8.0 min unit
11.3 min integration
12.7 min deploy
--- findings ---
[gate-never-fires] docs-build
has never failed; it costs 3.1 min per run and there is no evidence it catch
[gate-never-fires] licence-scan
has never failed; it costs 2.4 min per run and there is no evidence it catch
[no-cache] docs-build
takes 3.1 min with nothing cached; a dependency cache is usually the single
[no-cache] install
takes 4.5 min with nothing cached; a dependency cache is usually the single
--- what removing each never-firing gate would buy ---
licence-scan NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 2.4
docs-build NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 3.1
--- what caching the install would buy instead ---
install 4.5 -> 0.8 min (a dependency cache)
wall 12.7 -> 9.0 min saves 3.7 min per run
new critical path: checkout -> install -> integration -> deploy
deleting BOTH never-firing gates saved 0.0 min of wall clock;
one cache on the critical path saves 3.7.
--- the same jobs, triggered unsafely ---
[secrets-exposed-to-forks] deploy
pull_request_target runs with repository secrets against code the fork contr
findings of this kind: 1
measured-values.txt
Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.
python3 examples/pipeline_demo.py -> 35 lines, exit 0
bash tests/run_tests.sh -> 27 passed in 0.01s
the untouched starter -> 27 failed in 0.08s
One CI pipeline of eight jobs, of the kind that grows a stage at a time.
wall clock 12.7 min checkout -> install -> integration -> deploy
runner time 22.2 min billed
the gap 9.5 min of parallel work
Two numbers, optimised by different actions. The person waiting cares about the
first; the person paying cares about the second.
The review finds four things:
gate-never-fires docs-build 3.1 min per run, never failed
gate-never-fires licence-scan 2.4 min per run, never failed
no-cache docs-build 3.1 min with nothing cached
no-cache install 4.5 min with nothing cached
The obvious move is to delete the two jobs that have never caught anything.
Here is what that buys:
licence-scan NOT on the critical path wall 12.7 -> 12.7 saves 0.0 min
docs-build NOT on the critical path wall 12.7 -> 12.7 saves 0.0 min
Zero. Nothing was waiting for either of them, so removing them cannot make the
pipeline finish sooner. It saves 5.5 runner minutes per run, which is real money
and is not what anybody was complaining about.
What does make it faster:
install 4.5 -> 0.8 min (a dependency cache)
wall 12.7 -> 9.0 min saves 3.7 min per run
One cache, on the one job that everything else waits for.
deleting BOTH never-firing gates: 0.0 min of wall clock
one cache on the critical path: 3.7 min
That is the finding. To make a pipeline FASTER, shorten the critical path. To
make it CHEAPER, remove work anywhere. The instinct to delete the stage that
looks pointless addresses the second while the complaint is usually about the
first.
The security finding, on the same jobs with a different trigger:
secrets-exposed-to-forks deploy
pull_request_target runs with repository secrets in the context of the base
repository. Combined with a checkout of the fork's code -- which is what people
add next, because otherwise the workflow cannot see the change -- untrusted code
runs with your secrets in the environment. Plain pull_request has a read-only
token and no secrets, and the lab deliberately does not report it, because a
rule that fires on the safe case trains people to ignore the finding that
matters.
One implementation note. `without` re-points dependants: when a job is removed,
anything that needed it inherits its dependencies. An earlier version simply
dropped the edge, which turned dependants into root jobs that start at zero and
overstated the wall-clock saving of every deletion.
run-output.txt
--- the pipeline as it stands ---
wall clock 12.7 min along checkout -> install -> integration -> deploy
runner time 22.2 min billed
the gap 9.5 min of work happening in parallel
--- when each job finishes ---
0.3 min checkout
4.8 min install
5.6 min lint
7.2 min licence-scan
7.9 min docs-build
8.0 min unit
11.3 min integration
12.7 min deploy
--- findings ---
[gate-never-fires] docs-build
has never failed; it costs 3.1 min per run and there is no evidence it catch
[gate-never-fires] licence-scan
has never failed; it costs 2.4 min per run and there is no evidence it catch
[no-cache] docs-build
takes 3.1 min with nothing cached; a dependency cache is usually the single
[no-cache] install
takes 4.5 min with nothing cached; a dependency cache is usually the single
--- what removing each never-firing gate would buy ---
licence-scan NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 2.4
docs-build NOT on the critical path wall 12.7 -> 12.7 (saves 0.0) runner minutes saved 3.1
--- what caching the install would buy instead ---
install 4.5 -> 0.8 min (a dependency cache)
wall 12.7 -> 9.0 min saves 3.7 min per run
new critical path: checkout -> install -> integration -> deploy
deleting BOTH never-firing gates saved 0.0 min of wall clock;
one cache on the critical path saves 3.7.
--- the same jobs, triggered unsafely ---
[secrets-exposed-to-forks] deploy
pull_request_target runs with repository secrets against code the fork contr
findings of this kind: 1
starter-run.txt
An untouched starter fails the suite. Captured from a real run:
FAILED tests/test_pipeline.py::test_a_slow_job_with_a_cache_is_not_reported
FAILED tests/test_pipeline.py::test_a_quick_job_without_a_cache_is_left_alone
FAILED tests/test_pipeline.py::test_a_cycle_suppresses_the_other_findings - N...
FAILED tests/test_pipeline.py::test_pull_request_target_with_secrets_is_reported
FAILED tests/test_pipeline.py::test_the_same_job_on_pull_request_is_not_reported
FAILED tests/test_pipeline.py::test_pull_request_target_without_secrets_is_not_reported
FAILED tests/test_pipeline.py::test_removing_a_job_drops_it - NotImplementedE...
FAILED tests/test_pipeline.py::test_removing_a_job_does_not_orphan_its_dependants
FAILED tests/test_pipeline.py::test_removing_a_job_that_does_not_exist_is_an_error
FAILED tests/test_pipeline.py::test_removing_an_off_path_job_saves_runner_minutes_but_no_wall_clock
FAILED tests/test_pipeline.py::test_removing_an_on_path_job_does_save_wall_clock
27 failed in 0.08s
Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.
test-output.txt
........................... [100%]
27 passed in 0.01s
test-run.txt
........................... [100%]
27 passed in 0.01s
Source files
examples/pipeline_demo.py (3734 bytes)
#!/usr/bin/env python3
"""One CI pipeline, read three ways: how long, how safe, how useful."""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from pipeline import (
Job,
Trigger,
Workflow,
critical_path,
finish_times,
review,
savings_from_removing,
total_minutes,
)
# A pipeline that grew one stage at a time, as they all do.
CURRENT = Workflow(
"ci",
triggers=[Trigger.PUSH, Trigger.PULL_REQUEST],
jobs={
"checkout": Job("checkout", [], 0.3, caches=["git"], fails_per_100=0.1),
"install": Job("install", ["checkout"], 4.5, fails_per_100=0.4),
"lint": Job("lint", ["install"], 0.8, caches=["pip"], fails_per_100=6.0),
"unit": Job("unit", ["install"], 3.2, caches=["pip"], fails_per_100=9.0),
"integration": Job("integration", ["install"], 6.5, caches=["pip"], fails_per_100=2.0),
"licence-scan": Job("licence-scan", ["install"], 2.4, fails_per_100=0.0),
"docs-build": Job("docs-build", ["install"], 3.1, fails_per_100=0.0),
"deploy": Job("deploy", ["unit", "integration", "lint"], 1.4,
uses_secrets=True, fails_per_100=0.6),
},
)
# The same jobs, but triggered in the way that hands secrets to forks.
UNSAFE = Workflow(
"ci-unsafe",
triggers=[Trigger.PUSH, Trigger.PR_TARGET],
jobs=CURRENT.jobs,
)
def show(label: str, wf: Workflow) -> None:
path, wall = critical_path(wf)
print(f"--- {label} ---")
print(f" wall clock {wall} min along {' -> '.join(path)}")
print(f" runner time {total_minutes(wf)} min billed")
print(f" the gap {round(total_minutes(wf) - wall, 2)} min of work happening in parallel")
def main() -> int:
show("the pipeline as it stands", CURRENT)
print("--- when each job finishes ---")
for name, at in sorted(finish_times(CURRENT).items(), key=lambda kv: (kv[1], kv[0])):
print(f" {at:>5.1f} min {name}")
print("--- findings ---")
for f in review(CURRENT):
print(f" [{f.rule}] {f.job}")
print(f" {f.message[:76]}")
print("--- what removing each never-firing gate would buy ---")
for name in ("licence-scan", "docs-build"):
s = savings_from_removing(CURRENT, name)
on = "on" if s["on_critical_path"] else "NOT on"
print(f" {name:<14} {on} the critical path "
f"wall {s['wall_before']} -> {s['wall_after']} (saves {s['wall_saved']}) "
f"runner minutes saved {s['minutes_saved']}")
print("--- what caching the install would buy instead ---")
cached = Workflow(
CURRENT.name, list(CURRENT.triggers),
{n: (Job(j.name, list(j.needs), 0.8, j.uses_secrets, ["pip"], j.fails_per_100)
if n == "install" else j)
for n, j in CURRENT.jobs.items()},
)
_, before = critical_path(CURRENT)
path_after, after = critical_path(cached)
print(f" install 4.5 -> 0.8 min (a dependency cache)")
print(f" wall {before} -> {after} min saves {round(before - after, 2)} min per run")
print(f" new critical path: {' -> '.join(path_after)}")
print(f" deleting BOTH never-firing gates saved 0.0 min of wall clock;")
print(f" one cache on the critical path saves {round(before - after, 2)}.")
print("--- the same jobs, triggered unsafely ---")
unsafe = [f for f in review(UNSAFE) if f.rule == "secrets-exposed-to-forks"]
for f in unsafe:
print(f" [{f.rule}] {f.job}")
print(f" {f.message[:76]}")
print(f" findings of this kind: {len(unsafe)}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
examples/pipeline.py (7719 bytes)
"""Reason about a CI pipeline before you push and wait eleven minutes to find out.
Offline and standard-library only. No runner, no GitHub, no network. The input
is a described workflow -- jobs, what each needs, how long it takes, how often
it fails -- and the questions worth asking are answerable from that.
Three of them:
how long the critical path, not the sum. Independent jobs run in parallel.
how safe which jobs can reach a secret, and whether a fork can trigger them
how useful a gate that never fails is not a gate
The third is the one that decays quietly. A test job that has not failed in six
months is either protecting nothing or being ignored, and the pipeline gets
slower every quarter as people add stages nobody has ever seen go red.
Nothing here runs your tests. It reasons about the SHAPE of a pipeline, which
is the part you can get wrong for months without noticing.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
class Trigger(str, Enum):
PUSH = "push" # a branch in your own repository
PULL_REQUEST = "pull_request" # read-only token, no secrets by default
PR_TARGET = "pull_request_target" # runs with SECRETS against fork code
SCHEDULE = "schedule"
MANUAL = "workflow_dispatch"
@dataclass
class Job:
name: str
needs: list[str] = field(default_factory=list)
minutes: float = 1.0
uses_secrets: bool = False
caches: list[str] = field(default_factory=list)
fails_per_100: float = 0.0 # how often this job catches something
@dataclass
class Workflow:
name: str
triggers: list[Trigger]
jobs: dict[str, Job]
def missing_needs(wf: Workflow) -> list[str]:
"""Dependencies naming a job the workflow does not define."""
return sorted(
f"{name} needs '{dep}', which no job defines"
for name, job in wf.jobs.items()
for dep in job.needs
if dep not in wf.jobs
)
def find_cycle(wf: Workflow) -> list[str]:
"""One dependency cycle if any exists, normalised so it is stable."""
def walk(node: str, path: list[str]) -> list[str] | None:
if node in path:
cycle = path[path.index(node):]
return list(min(tuple(cycle[i:] + cycle[:i]) for i in range(len(cycle))))
for dep in wf.jobs[node].needs if node in wf.jobs else []:
found = walk(dep, path + [node])
if found:
return found
return None
for name in sorted(wf.jobs):
found = walk(name, [])
if found:
return found
return []
def finish_times(wf: Workflow) -> dict[str, float]:
"""When each job finishes, assuming unlimited parallel runners.
A job starts as soon as everything it needs has finished, so its finish
time is the latest of its dependencies plus its own duration.
"""
if find_cycle(wf):
raise ValueError(f"dependency cycle: {' -> '.join(find_cycle(wf))}")
times: dict[str, float] = {}
remaining = dict(wf.jobs)
while remaining:
ready = [n for n, j in remaining.items()
if all(d in times or d not in wf.jobs for d in j.needs)]
if not ready:
raise ValueError("no job can start; check the dependency graph")
for name in sorted(ready):
job = remaining.pop(name)
start = max((times[d] for d in job.needs if d in times), default=0.0)
times[name] = round(start + job.minutes, 2)
return times
def critical_path(wf: Workflow) -> tuple[list[str], float]:
"""The longest chain of dependent jobs, and how long it takes.
This is what a pipeline actually costs in wall-clock time. Summing the job
durations overstates it, sometimes by a lot, because independent jobs run
at the same time.
"""
times = finish_times(wf)
if not times:
return [], 0.0
end = max(times, key=lambda n: (times[n], n))
chain = [end]
while wf.jobs[chain[-1]].needs:
deps = [d for d in wf.jobs[chain[-1]].needs if d in times]
if not deps:
break
chain.append(max(deps, key=lambda d: (times[d], d)))
return list(reversed(chain)), round(times[end], 2)
def total_minutes(wf: Workflow) -> float:
"""Runner minutes consumed. What you are billed for, and not what you wait."""
return round(sum(j.minutes for j in wf.jobs.values()), 2)
@dataclass(frozen=True)
class Finding:
rule: str
job: str
message: str
def review(wf: Workflow) -> list[Finding]:
"""Everything wrong with this pipeline's shape."""
out: list[Finding] = []
for msg in missing_needs(wf):
out.append(Finding("undefined-need", msg.split()[0], msg))
cycle = find_cycle(wf)
if cycle:
out.append(
Finding("dependency-cycle", cycle[0],
f"cycle: {' -> '.join(cycle + [cycle[0]])}; no job in it can run")
)
return out
# pull_request_target runs with repository secrets against code from a fork.
if Trigger.PR_TARGET in wf.triggers:
for name, job in sorted(wf.jobs.items()):
if job.uses_secrets:
out.append(
Finding("secrets-exposed-to-forks", name,
"pull_request_target runs with repository secrets against code the "
"fork controls; a pull request can exfiltrate them")
)
# A gate that has never failed is not protecting anything you can point to.
for name, job in sorted(wf.jobs.items()):
if job.fails_per_100 == 0.0 and job.minutes > 0:
out.append(
Finding("gate-never-fires", name,
f"has never failed; it costs {job.minutes} min per run and there is no "
"evidence it catches anything")
)
# A slow job with no cache is the cheapest thing to fix.
for name, job in sorted(wf.jobs.items()):
if job.minutes >= 3.0 and not job.caches:
out.append(
Finding("no-cache", name,
f"takes {job.minutes} min with nothing cached; a dependency cache is "
"usually the single largest saving available")
)
return out
def without(wf: Workflow, job_name: str) -> Workflow:
"""The same workflow with one job removed, and dependants re-pointed.
Removing a job must not orphan the jobs that needed it -- they inherit its
dependencies, which is what actually happens when you delete a stage.
"""
if job_name not in wf.jobs:
raise KeyError(job_name)
inherited = wf.jobs[job_name].needs
jobs = {}
for name, job in wf.jobs.items():
if name == job_name:
continue
needs = [d for d in job.needs if d != job_name]
if job_name in job.needs:
needs += [d for d in inherited if d not in needs]
jobs[name] = Job(job.name, needs, job.minutes, job.uses_secrets,
list(job.caches), job.fails_per_100)
return Workflow(wf.name, list(wf.triggers), jobs)
def savings_from_removing(wf: Workflow, job_name: str) -> dict[str, float]:
"""What deleting one job buys in wall-clock and in runner minutes."""
before_path, before_wall = critical_path(wf)
after = without(wf, job_name)
_, after_wall = critical_path(after)
return {
"wall_before": before_wall,
"wall_after": after_wall,
"wall_saved": round(before_wall - after_wall, 2),
"minutes_saved": round(total_minutes(wf) - total_minutes(after), 2),
"on_critical_path": job_name in before_path,
}
metadata.yml (999 bytes)
lesson_id: D335
day: 335
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python3 examples/pipeline_demo.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3 examples/pipeline_demo.py
reproduced the documented figures exactly (12.7 min wall clock against 22.2 runner minutes; deleting
both never-firing gates saving 0.0 min of wall clock while one cache on the critical path saves 3.7;
and pull_request_target reporting the deploy job as exposed); bash tests/run_tests.sh -> 27 passed in
0.01s. The untouched starter fails all 27. requires_network is false and NO GitHub account or runner
is used: the shape of a pipeline is what you can get wrong for months, and it is checkable offline.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/pipeline.py (4353 bytes)
"""STARTER -- implement each TASK below, then run the tests.
Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
Reason about a CI pipeline before you push and wait eleven minutes to find out.
Offline and standard-library only. No runner, no GitHub, no network. The input
is a described workflow -- jobs, what each needs, how long it takes, how often
it fails -- and the questions worth asking are answerable from that.
Three of them:
how long the critical path, not the sum. Independent jobs run in parallel.
how safe which jobs can reach a secret, and whether a fork can trigger them
how useful a gate that never fails is not a gate
The third is the one that decays quietly. A test job that has not failed in six
months is either protecting nothing or being ignored, and the pipeline gets
slower every quarter as people add stages nobody has ever seen go red.
Nothing here runs your tests. It reasons about the SHAPE of a pipeline, which
is the part you can get wrong for months without noticing.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
class Trigger(str, Enum):
PUSH = 'push'
PULL_REQUEST = 'pull_request'
PR_TARGET = 'pull_request_target'
SCHEDULE = 'schedule'
MANUAL = 'workflow_dispatch'
@dataclass
class Job:
name: str
needs: list[str] = field(default_factory=list)
minutes: float = 1.0
uses_secrets: bool = False
caches: list[str] = field(default_factory=list)
fails_per_100: float = 0.0
@dataclass
class Workflow:
name: str
triggers: list[Trigger]
jobs: dict[str, Job]
def missing_needs(wf: Workflow) -> list[str]:
"""Dependencies naming a job the workflow does not define."""
raise NotImplementedError('TASK 1: implement missing_needs. Dependencies naming a job the workflow does not define.')
def find_cycle(wf: Workflow) -> list[str]:
"""One dependency cycle if any exists, normalised so it is stable."""
raise NotImplementedError('TASK 3: implement find_cycle. One dependency cycle if any exists, normalised so it is stable.')
def finish_times(wf: Workflow) -> dict[str, float]:
"""When each job finishes, assuming unlimited parallel runners.
A job starts as soon as everything it needs has finished, so its finish
time is the latest of its dependencies plus its own duration.
"""
raise NotImplementedError('TASK 4: implement finish_times. When each job finishes, assuming unlimited parallel runners.')
def critical_path(wf: Workflow) -> tuple[list[str], float]:
"""The longest chain of dependent jobs, and how long it takes.
This is what a pipeline actually costs in wall-clock time. Summing the job
durations overstates it, sometimes by a lot, because independent jobs run
at the same time.
"""
raise NotImplementedError('TASK 5: implement critical_path. The longest chain of dependent jobs, and how long it takes.')
def total_minutes(wf: Workflow) -> float:
"""Runner minutes consumed. What you are billed for, and not what you wait."""
raise NotImplementedError('TASK 6: implement total_minutes. Runner minutes consumed. What you are billed for, and not what you wait.')
@dataclass(frozen=True)
class Finding:
rule: str
job: str
message: str
def review(wf: Workflow) -> list[Finding]:
"""Everything wrong with this pipeline's shape."""
raise NotImplementedError("TASK 7: implement review. Everything wrong with this pipeline's shape.")
def without(wf: Workflow, job_name: str) -> Workflow:
"""The same workflow with one job removed, and dependants re-pointed.
Removing a job must not orphan the jobs that needed it -- they inherit its
dependencies, which is what actually happens when you delete a stage.
"""
raise NotImplementedError('TASK 8: implement without. The same workflow with one job removed, and dependants re-pointed.')
def savings_from_removing(wf: Workflow, job_name: str) -> dict[str, float]:
"""What deleting one job buys in wall-clock and in runner minutes."""
raise NotImplementedError('TASK 9: implement savings_from_removing. What deleting one job buys in wall-clock and in runner minutes.')
tests/run_tests.sh (210 bytes)
#!/usr/bin/env bash
# Run the day 335 CI pipeline test suite.
# Offline, standard library plus pytest. No runner and no GitHub account.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_pipeline.py (6537 bytes)
"""Grouped by what the pipeline analysis decides.
Run with: bash tests/run_tests.sh
"""
from __future__ import annotations
import os
import sys
import pytest
HERE = os.path.dirname(os.path.abspath(__file__))
sys.path.insert(0, os.path.join(os.path.dirname(HERE), "examples"))
from pipeline import ( # noqa: E402
Job,
Trigger,
Workflow,
critical_path,
find_cycle,
finish_times,
missing_needs,
review,
savings_from_removing,
total_minutes,
without,
)
def wf(jobs, triggers=(Trigger.PUSH,)):
return Workflow("ci", list(triggers), {j.name: j for j in jobs})
def job(name, needs=(), minutes=1.0, secrets=False, caches=(), fails=1.0):
return Job(name, list(needs), minutes, secrets, list(caches), fails)
# ------------------------------------------------------------------ timing
def test_a_single_job_finishes_after_its_own_duration():
assert finish_times(wf([job("a", minutes=2.5)])) == {"a": 2.5}
def test_dependent_jobs_add_up():
t = finish_times(wf([job("a", minutes=2.0), job("b", ["a"], minutes=3.0)]))
assert t["b"] == 5.0
def test_independent_jobs_run_in_parallel_not_in_sequence():
# The whole reason wall clock and runner minutes differ.
t = finish_times(wf([job("a", minutes=2.0), job("b", minutes=3.0)]))
assert t["a"] == 2.0 and t["b"] == 3.0
def test_a_job_waits_for_its_slowest_dependency():
t = finish_times(wf([
job("fast", minutes=1.0), job("slow", minutes=5.0),
job("after", ["fast", "slow"], minutes=1.0),
]))
assert t["after"] == 6.0
def test_a_dependency_on_an_undefined_job_does_not_block_timing():
t = finish_times(wf([job("a", ["nowhere"], minutes=2.0)]))
assert t["a"] == 2.0
# ------------------------------------------------------------ critical path
def test_the_critical_path_is_the_longest_chain_not_the_sum():
w = wf([job("a", minutes=2.0), job("b", ["a"], minutes=3.0), job("side", minutes=10.0)])
path, wall = critical_path(w)
assert wall == 10.0 and path == ["side"]
assert total_minutes(w) == 15.0
def test_runner_minutes_exceed_wall_clock_when_work_is_parallel():
w = wf([job("a", minutes=4.0), job("b", minutes=4.0), job("c", minutes=4.0)])
_, wall = critical_path(w)
assert wall == 4.0
assert total_minutes(w) == 12.0
def test_the_critical_path_is_a_real_chain_of_dependencies():
w = wf([job("a", minutes=1.0), job("b", ["a"], minutes=2.0), job("c", ["b"], minutes=3.0)])
path, wall = critical_path(w)
assert path == ["a", "b", "c"] and wall == 6.0
def test_an_empty_workflow_has_no_critical_path():
assert critical_path(wf([])) == ([], 0.0)
# ------------------------------------------------------------------ graph
def test_a_cycle_is_detected():
assert find_cycle(wf([job("a", ["b"]), job("b", ["a"])])) == ["a", "b"]
def test_an_acyclic_graph_has_no_cycle():
assert find_cycle(wf([job("a"), job("b", ["a"])])) == []
def test_timing_a_cyclic_workflow_raises():
with pytest.raises(ValueError, match="cycle"):
finish_times(wf([job("a", ["b"]), job("b", ["a"])]))
def test_an_undefined_dependency_is_reported():
assert missing_needs(wf([job("a", ["ghost"])])) == ["a needs 'ghost', which no job defines"]
# ------------------------------------------------------------------ review
def test_a_gate_that_has_never_failed_is_reported():
w = wf([job("scan", minutes=2.0, fails=0.0)])
assert "gate-never-fires" in {f.rule for f in review(w)}
def test_a_gate_that_does_fail_is_not_reported():
w = wf([job("unit", minutes=2.0, caches=["pip"], fails=5.0)])
assert "gate-never-fires" not in {f.rule for f in review(w)}
def test_a_slow_job_with_no_cache_is_reported():
assert "no-cache" in {f.rule for f in review(wf([job("install", minutes=4.0)]))}
def test_a_slow_job_with_a_cache_is_not_reported():
w = wf([job("install", minutes=4.0, caches=["pip"])])
assert "no-cache" not in {f.rule for f in review(w)}
def test_a_quick_job_without_a_cache_is_left_alone():
assert "no-cache" not in {f.rule for f in review(wf([job("tiny", minutes=0.5)]))}
def test_a_cycle_suppresses_the_other_findings():
w = wf([job("a", ["b"], minutes=9.0, fails=0.0), job("b", ["a"], minutes=9.0, fails=0.0)])
rules = {f.rule for f in review(w)}
assert rules == {"dependency-cycle"}
# ---------------------------------------------------------------- secrets
def test_pull_request_target_with_secrets_is_reported():
w = wf([job("deploy", minutes=1.0, secrets=True)],
triggers=(Trigger.PUSH, Trigger.PR_TARGET))
assert "secrets-exposed-to-forks" in {f.rule for f in review(w)}
def test_the_same_job_on_pull_request_is_not_reported():
w = wf([job("deploy", minutes=1.0, secrets=True)],
triggers=(Trigger.PUSH, Trigger.PULL_REQUEST))
assert "secrets-exposed-to-forks" not in {f.rule for f in review(w)}
def test_pull_request_target_without_secrets_is_not_reported():
w = wf([job("build", minutes=1.0, secrets=False)],
triggers=(Trigger.PR_TARGET,))
assert "secrets-exposed-to-forks" not in {f.rule for f in review(w)}
# ---------------------------------------------------------------- removal
def test_removing_a_job_drops_it():
w = without(wf([job("a"), job("b")]), "a")
assert set(w.jobs) == {"b"}
def test_removing_a_job_does_not_orphan_its_dependants():
# b needed a, a needed checkout. Remove a and b must still wait for checkout.
w = without(wf([job("checkout"), job("a", ["checkout"]), job("b", ["a"])]), "a")
assert w.jobs["b"].needs == ["checkout"]
def test_removing_a_job_that_does_not_exist_is_an_error():
with pytest.raises(KeyError):
without(wf([job("a")]), "nope")
def test_removing_an_off_path_job_saves_runner_minutes_but_no_wall_clock():
# THE finding: deleting a slow job that nothing waits for makes the
# pipeline cheaper and not one second faster.
w = wf([
job("install", minutes=4.0),
job("long", ["install"], minutes=6.0),
job("side", ["install"], minutes=3.0, fails=0.0),
])
s = savings_from_removing(w, "side")
assert s["on_critical_path"] is False
assert s["wall_saved"] == 0.0
assert s["minutes_saved"] == 3.0
def test_removing_an_on_path_job_does_save_wall_clock():
w = wf([job("install", minutes=4.0), job("long", ["install"], minutes=6.0)])
s = savings_from_removing(w, "long")
assert s["on_critical_path"] is True
assert s["wall_saved"] == 6.0
Troubleshooting
Troubleshooting — Day 335
NotImplementedError on every test
Expected. The starter stubs nine functions — see expected-output/starter-run.txt.
Start with finish_times; the critical path, the review and the savings all read from it.
test_independent_jobs_run_in_parallel_not_in_sequence fails
You are accumulating durations across every job rather than only along dependency edges. A job's finish time is max(finish of its needs) + its own duration, and a job with no needs starts at zero regardless of what else exists.
If this is wrong, wall clock equals runner minutes everywhere and the entire lesson disappears.
test_the_critical_path_is_the_longest_chain_not_the_sum fails
The critical path is the chain ending at the latest-finishing job, walked backwards through whichever dependency finished last. It is not the sum of all durations, and it is not necessarily the longest chain by number of jobs.
finish_times loops forever
A cycle. Check for one and raise before the scheduling loop, rather than spinning waiting for a job that can never become ready.
test_a_cycle_suppresses_the_other_findings fails
review must return the cycle finding and stop. With no valid ordering there are no finish times, so every timing-derived finding would either be wrong or raise. A fatal finding should suppress the derived ones.
test_removing_a_job_does_not_orphan_its_dependants fails
This is the subtle one. When you delete job a, anything that needed a must inherit a's own dependencies — otherwise b becomes a root job and starts immediately, which silently understates the wall clock and makes deletion look better than it is.
savings_from_removing reports wall-clock savings for an off-path job
Then your critical path is wrong, because removing a job nothing waits for cannot make the pipeline finish sooner. Getting 0.0 here is the correct and instructive answer.
secrets-exposed-to-forks fires on pull_request
Only pull_request_target hands repository secrets to a workflow running against fork-controlled code. Plain pull_request runs with a read-only token and no secrets, and reporting it produces noise that trains people to ignore the finding that matters.
no-cache fires on everything
The rule is for jobs that are slow — three minutes or more — and have no cache declared. A half-minute job without a cache is not worth a finding, and a rule that fires on every job is a rule nobody reads.
The numbers do not match my pipeline
They will not. The durations and failure rates in the demo are illustrative. Put your own in: you can read job durations from any recent run, and the failure rates from how often each job has actually gone red this quarter — which is a number most teams have never looked at.
Security notes
Security notes — Day 335
What this lab does and does not touch
- No GitHub account, no runner, no workflow is executed. The input is a described pipeline. Nothing is dispatched and no token is read.
- No network, no credentials.
- No writes outside this directory. Only
__pycache__/,.pytest_cache/and, if you create it,.venv/. - No
sudo, no global installs.
pull_request_target is the one to understand
This is the single most dangerous line in GitHub Actions, and it is dangerous because it looks like a small variation on a safe thing.
pull_requestchecks out the fork's code and runs with a read-only token and no secrets. A hostile pull request can run whatever it likes and gain nothing.pull_request_targetruns in the context of the base repository — with your secrets available — and exists so that a workflow can label or comment on a PR from a fork.
The trap is combining it with a checkout of the fork's code. At that point untrusted code executes with your repository secrets in the environment, and exfiltrating them is a one-line change to a test file or a Makefile. This has been used against real projects repeatedly.
The rule that survives: pull_request_target may inspect a pull request; it must never run its code. If you need both, split into two workflows — one that runs untrusted code with no secrets, one that reacts to the result.
The lab reports every secret-using job under a pull_request_target trigger for this reason.
Least privilege for the token
GITHUB_TOKEN defaults to broad permissions in older repositories. Set them explicitly, at the top of the workflow and narrowed per job:
permissions:
contents: read
Then grant only what a specific job needs. A lint job needs nothing; a job that pushes a release needs contents: write and nothing else.
Third-party actions are supply chain
uses: some-org/some-action@v3 executes somebody else's code with your token. A tag is mutable — the author, or anyone who compromises them, can move v3. Pin to a full commit SHA for anything that touches secrets, and read what you are pinning.
This is Day 347's subject and it belongs here too, because CI is where most organisations run the most untrusted code with the most privilege.
Caches are writable by the jobs that read them
A cache is keyed content that later runs restore and trust. A workflow that can be triggered from a fork and writes to a shared cache can poison what a later privileged run restores. Scope cache keys so untrusted runs cannot write to a key a trusted run reads.
A gate that never fails is a security problem too
The lab reports jobs that have never failed. The framing is usually about time, and there is a security version: a scanner that has never reported anything is either configured wrongly or nobody reads its output. Both mean the control does not exist while appearing on the diagram.
Reversing everything this lab did
rm -rf __pycache__ .pytest_cache .venv
Nothing else was created, and nothing outside this directory was modified.