Computing Foundations › Systems Foundations: Storage, Observability, and Tooling › Day 41

Day 41: Thinking in Automation: Scripts, Hooks, and Pipelines

Day 41 of 365 — Thinking in Automation: Scripts, Hooks, and Pipelines

After this lesson you will be able to decide when a task is worth automating, climb the ladder from a shell script to a git hook to a CI pipeline, and install a real pre-commit quality gate that fails fast — the connective tissue that makes everything else you have learned dependable.

Course
Computing Foundations
Category
Systems Foundations: Storage, Observability, and Tooling
Reading time
≈ 40 min
Practical time
≈ 30 min
Lesson duration
1h 10m
Last verified
2026-07-12

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-041-thinking-in-automation-scripts-hooks-and

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/computing-foundations/day-041-thinking-in-automation-scripts-hooks-and
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters


The idea in plain language

Diagram: time-triggered automation compared with event-triggered automation


Historical background


What it is — and what it is not

Automation is:

It is not:


Why it was created and what problems it solves


How it works

The ladder, rung by rung

Diagram: how the exit code connects each rung of the automation ladder

Diagram: the automation ladder, from manual work up through scripts, scheduling, hooks, and a CI pipeline, with increasing leverage

The pipeline

Diagram: a pipeline ordered cheapest-first compared with expensive-first

Flowchart: a CI pipeline where a push triggers lint, then test, then build, then deploy, with a failing gate stopping the line before deploy

Three properties make a pipeline trustworthy:

Secrets in automation

Diagram: a hard-coded secret compared with one supplied through the environment

Making it observable

Diagram: a local hook and a CI check as two layers of the same defence

Diagram: the stages by which an unwatched automated job fails unnoticed


An everyday analogy

Diagram: how a dependency graph rebuilds only what changed

A factory production line.

The factoryAutomation
A written work instructionA script
The shift bellA schedule
A sensor that stops the line on a defectA hook
The line itself, station by stationA pipeline
Quick visual check first, load test lastCheap checks first
The line halting on a faultFail fast
The shift logLogging

Examples in practice

Diagram: fail fast, idempotency and reproducibility in a pipeline

Diagram: a pre-commit hook compared with the same check in continuous integration


Implications: security, privacy, performance, scalability, and cost


Alternatives: free, open source, and commercial

ToolTypeWhere it fits
Shell scriptsFree, built inThe foundation. Everything above reuses them.
cron / systemd timersFree, built inTime-triggered work on one machine
pre-commitFree, open sourceManaging hooks consistently across a team
GitHub Actions, GitLab CIFree tiersEvent-triggered checks, for everyone, unskippable
make, justFree, open sourceLocal task running with dependency ordering
Airflow, Dagster, PrefectFree, open sourceData pipelines with dependencies, retries and backfills

TermWhat it actually is
ScriptCommands in a file, run top to bottom
ScheduleA trigger based on time
HookA trigger based on an event
PipelineOrdered stages, each gating the next
CIChecks run on every push, for everyone
CDPackaging and releasing automatically when they pass
IdempotentSafe to run more than once

When to use it — and when not to

Diagram: tasks worth automating compared with tasks better done by hand


The AI connection


Knowledge check

quiz:
  question: "A team has a pre-commit hook that blocks commits containing credentials. A secret still reaches the shared repository. What is the most likely explanation?"
  options:
    - "Hooks are local and skippable, so a teammate without it installed — or using --no-verify — bypassed the check entirely"
    - "The hook only inspects staged files, and the secret was committed unstaged"
    - "The hook ran but its non-zero exit status was ignored by git"
    - "Pre-commit hooks cannot inspect file contents, only file names"
  answer_index: 0
  explanation: "A hook lives in the local .git directory, does not travel with a clone, and can be bypassed with a flag. That is precisely why the same check belongs in CI as well: the hook is fast feedback for you, and CI is the gate that applies to everyone."

Hands-on exercise

Climb the ladder on one real task. Worked through fully in the Day 41 lab directory.

What you are doingCommand
Write the scriptcheck.sh, with set -euo pipefail from Day 12
Run it by hand./check.sh — confirm it exits 0 and non-zero correctly
Schedule itA crontab line, with output redirected and 2>&1
Make it a hook.git/hooks/pre-commit, then chmod +x
Prove the hook blocksTry to commit something it rejects
Prove it is skippablegit commit --no-verify
Add cheap-first stagesLint, then test, then build — in that order

Expected output

$ ./check.sh; echo "exit=$?"
lint: ok
test: 12 passed
exit=0

$ git commit -m "add key"
pre-commit: possible credential in config.py — rejected
$ git commit -m "add key" --no-verify
[main 4f2a1c8] add key

Validate your work

Troubleshooting

Common mistakes


Practice assignment


Extension challenge


Quiz

Q1. According to the common heuristic in this lesson, when is the clearest signal that a task is worth turning into a script?

  1. The first time you ever perform it
  2. Only after a manager asks you to
  3. By the third time you find yourself repeating it
  4. Never — every task should be scripted before it is done once
Show answer

Answer: C. By the third time you find yourself repeating it

The heuristic is "do it once, just do it; do it twice, note the steps; by the third time, script it." The third repetition signals the task is stable and frequent enough that automating it will pay back the cost of building it.

Q2. What triggers a git hook, as opposed to a cron job?

  1. A clock reaching a scheduled time
  2. A user logging into the machine
  3. A change in the amount of free disk space
  4. A git event such as making a commit or pushing
Show answer

Answer: D. A git event such as making a commit or pushing

A git hook is event-driven: it fires on a git action like a commit (pre-commit) or a push (pre-push). A cron job is time-driven, firing when the clock matches its schedule. That difference is why hooks suit checks-on-change and cron suits time-based jobs.

Q3. In a lint → test → build → deploy pipeline, what does "failing fast" mean?

  1. Stopping the moment a stage fails, so later stages never run
  2. Skipping the slow stages to save time
  3. Running all four stages as quickly as possible in parallel
  4. Deploying first and running the checks afterward
Show answer

Answer: A. Stopping the moment a stage fails, so later stages never run

Failing fast means the pipeline halts at the first failing stage and reports immediately. Because the cheap, fast checks run first, most mistakes are caught in the first second and the expensive later stages never run on code that was never going to pass.

Q4. Why does a healthy project run both a local pre-commit hook and a CI pipeline, even when they run the same checks?

  1. CI is decorative; the hook does the real work
  2. The hook is fast but skippable; CI runs for everyone
  3. Running the checks twice makes them more accurate
  4. Hooks work only on macOS and CI works only on Linux
Show answer

Answer: B. The hook is fast but skippable; CI runs for everyone

A local hook is fast and private but skippable (with --no-verify) and only protects developers who installed it. CI runs on a shared server for every change from everyone and cannot be quietly bypassed, so it is the authoritative gate. The two are complementary lines of defense.

Q5. What is the correct way to give an automated deploy script an API token?

  1. Hard-code the token as a variable at the top of the script
  2. Commit the token to the repository so the script can always find it
  3. Pass it via an environment variable from a secret store
  4. Email the token to the whole team before each run
Show answer

Answer: C. Pass it via an environment variable from a secret store

Secrets must never be hard-coded or committed, because git history is permanent and copied to every clone. The script should read the secret by name from the environment, with the value stored in an access-controlled secret store or a git-ignored file.

Q6. What does it mean for a pipeline stage to be idempotent?

  1. It runs faster the second time
  2. It automatically deploys to production
  3. It can only ever be run once
  4. Running it twice gives the same result
Show answer

Answer: D. Running it twice gives the same result

Idempotency means re-running a stage on the same input yields the same result and leaves no state that changes the next run. A stage that appends to a file instead of overwriting it is not idempotent and will eventually surprise you; idempotency is what makes automation safe to re-run.

Q7. Which situation is the best candidate to leave manual rather than automate?

  1. A rare task whose steps change every time
  2. A nightly database backup
  3. A check that should run before every single commit
  4. Running the test suite on every push
Show answer

Answer: A. A rare task whose steps change every time

Automation pays off on repeated tasks with stable, well-understood steps. A task done rarely, whose steps keep changing, turns its automation into a maintenance burden that breaks as fast as you fix it — so it is better left manual until it stabilizes.

Q8. A machine-learning workflow — pull data, preprocess, train, evaluate, deploy — is best understood as what?

  1. A single command that cannot be broken into steps
  2. An ordered pipeline where evaluate gates deploy
  3. A purely manual process that must be redone by hand each time
  4. A cron job that runs once a year
Show answer

Answer: B. An ordered pipeline where evaluate gates deploy

An ML workflow is a pipeline with ordered stages and gates, exactly like lint-test-build-deploy. The evaluate stage is a fail-fast gate: if the new model scores worse, the pipeline stops and refuses to deploy it, protecting production the way a failing test stage protects a release.

Glossary

automation
The practice of encoding a repeatable task so a computer performs it without manual intervention, on a defined trigger, producing the same result every time.
script
A file of commands that runs top to bottom, capturing a sequence of steps so they can be replayed identically — the foundation of all automation.
cron
A background service that runs commands on a repeating schedule, defined by a crontab line with five time fields: minute, hour, day of month, month, and day of week.
git hook
A script that git runs automatically at a point in its workflow, such as before a commit; a hook only takes effect if it is marked executable.
pre-commit hook
A git hook that runs before a commit is recorded and can reject the commit by exiting with a non-zero status, acting as a local quality gate.
CI/CD
Continuous integration and continuous delivery/deployment: a shared server that automatically runs checks on every change (CI) and, when they pass, packages and releases the result (CD).
pipeline
A task broken into ordered stages where each stage must pass before the next begins, typically lint then test then build then deploy.
stage
One step in a pipeline that runs a specific check or action, such as linting or testing, and must succeed for the pipeline to continue.
fail fast
Stopping a pipeline the moment a stage fails, so no later stage runs on a known-bad input and the failure is reported immediately.
idempotency
The property that running an operation twice on the same input produces the same result, with no leftover state leaking between runs, making automation safe to re-run.
reproducibility
The property that the same inputs and environment produce the same output every time and everywhere, so an automated result can be trusted.
task runner
A tool such as make that gives project tasks memorable names and declares their order, so a hook, a CI job, and a developer can all invoke the same steps.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.