Computing Foundations › The Command Line › Day 14
Day 14: Automating Tasks with Shell Scripts and cron
After this lesson you will be able to turn a repetitive chore into a safe, logged shell script and read a cron schedule field by field — so you can decide what to automate, when to run it, and how to keep it trustworthy while nobody is watching.
Hands-on lab for this lesson
Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-014-automating-tasks-with-shell-scripts-and
- Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git cd ai-roadmap-365.github.io - Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
cd labs/sections/computing-foundations/day-014-automating-tasks-with-shell-scripts-and - Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
- Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
bash tests/run_tests.sh # or the test command named in the lab README
You can also open the lab as a local page (works offline, shows the file tree and expected output).
Learning objectives
By the end of this lesson you will be able to:
- Explain what automation is — a script plus a scheduler — and identify which tasks are worth automating (repetitive, scheduled, error-prone)
- Decode any five-field cron schedule (minute, hour, day-of-month, month, day-of-week) and write lines for common cases such as "8 p.m. every day" and "9 a.m. every Monday"
- Edit and inspect a personal schedule with crontab -e and crontab -l, and describe how the scheduler wakes each minute to match and run jobs
- Compare the leading schedulers — cron, macOS launchd, Linux systemd timers, and Windows Task Scheduler — and choose the right one for a platform
- Explain why unattended scripts must log their output, and make a script idempotent and dry-run-capable so it is safe to run twice and safe to preview
- Connect scheduled automation to real pipelines — scheduled data pulls, nightly training and evaluation runs, automated reports, and pipeline glue
Prerequisites
- Day 12 (shell scripting: variables, loops, and conditionals) — this lesson schedules the scripts you learned to write there
- A terminal on macOS, Linux, or WSL; no software to install, and no changes to your real crontab required
Why this matters
-
This is where the week pays off. Everything since Day 8 combines into work that happens without you.
-
Data pipelines run on schedules. Fetch overnight, preprocess at dawn, retrain weekly — all of it scheduled.
-
Unattended automation is a different discipline from interactive work. Nobody is watching, so the script must fail loudly and log everything.
-
The most common cron failure has nothing to do with cron. It is a script that assumed your environment and did not get it.
-
Today is also the last day of Week 2, and the point at which the terminal becomes leverage rather than a skill.
The idea in plain language
-
A scheduler is a program that watches the clock and starts commands when their time arrives.
-
cron is the classic one. Once a minute it compares the time against a list of rules and launches whatever matches.
-
It does not wait for your command to finish, and it does not show you the output.
-
Each rule is five time fields and a command, and the fields run from the smallest unit upward.
-
The hard part is not the schedule. It is writing a script that behaves correctly when nobody is watching.
Historical background
-
cron arrived in Version 7 Unix, 1979, and its syntax has barely changed since.
-
The original woke once a minute and read the whole table. Vixie cron, 1987, made it sleep until the next scheduled event instead, and that is what most Linux systems still run.
-
Output was emailed to the job’s owner, which was reasonable on a shared machine in 1979 and is why unattended jobs redirect to a log today.
-
macOS replaced cron with
launchdin 2005, though cron still works there. -
systemd timers arrived on Linux in the 2010s, adding logging, dependencies and catch-up for missed runs.
-
The five-field syntax outlived all of it, and appears now in Kubernetes, GitHub Actions and every CI system you will meet.
What it is — and what it is not
It is:
- A background program comparing the clock against a list of rules.
- A launcher: it starts your command as a new process and goes back to sleep.
- Indifferent to what happens next — it does not wait, watch, or retry.
It is not:
-
Not a workflow engine. It has no concept of one job depending on another.
-
Not a monitor. A job that fails every night will fail quietly for a year.
-
Not your shell. It reads none of your configuration, which is the source of most surprises.
-
Not exactly punctual on a laptop. A machine asleep at 3am simply misses the run; classic cron does not catch up.
Why it was created and what problems it solves
-
Work that must happen at a time nobody wants to be awake for. Backups, reports, cleanup.
-
Work that must happen regularly and identically. A human doing it weekly will do it differently each time.
-
Work whose value is in its regularity. A log rotated on schedule; a dataset refreshed before the morning.
-
Freeing attention. Anything scheduled is something you no longer have to remember.
-
Consistency across machines. The same crontab line on twenty servers does the same thing on all twenty.
How it works
Reading a crontab, field by field
crontab -eopens your schedule in an editor;crontab -llists it.- Each active line is five time fields, then a command.
| Position | Field | Values | * means |
|---|---|---|---|
| 1 | Minute | 0–59 | every minute |
| 2 | Hour | 0–23 | every hour |
| 3 | Day of month | 1–31 | every day |
| 4 | Month | 1–12 or jan | every month |
| 5 | Day of week | 0–7 (0 and 7 = Sunday) | every weekday |
- A list —
1,15— means the 1st and the 15th. - A range —
1-5— means 1 through 5. - A step —
*/15— means every fifteenth value. - So
*/15 9-17 * * 1-5is every 15 minutes, 9am to 5pm, weekdays.
From a run to a logged, safe job
-
The scheduler fires at the appointed minute and starts the script.
-
The script writes every meaningful action to a log, with a timestamp.
-
The exit status branches at the end: success is recorded and the job waits for next time; failure is recorded loudly, and in production would raise an alert.
-
If the work may already be partly done, the script checks and skips — so a second run is harmless.
-
Those two properties — logging and safety on repeat — are what make unattended automation trustworthy rather than alarming.
Redirecting the output
0 9 * * 1 /usr/local/bin/weekly-report.sh >> /var/log/weekly.log 2>&1
-
>>appends, so history survives. -
2>&1sends errors to the same place, so the log holds the whole story rather than the happy half. -
Without redirection, output is mailed or lost, and on a modern machine it is almost always lost.
An everyday analogy
An alarm clock, not an assistant.
| The alarm clock | cron |
|---|---|
| The time you set | The five fields |
| It rings whether you are there or not | The job starts regardless |
| It does not check you got up | No monitoring of the result |
| It does not ring again if the power was out | A missed run stays missed |
| A note beside the bed | The log file |
- The clock’s job ends the moment it rings. Everything after that is yours.
- An alarm you never check is worse than none, because you believe you are covered. That is a scheduled job with no logging.
Examples in practice
-
0 2 * * * backup.sh— nightly at 2am, when nobody is using the machine. -
*/5 * * * * healthcheck.sh— every five minutes, and the first job you should make safe against overlapping with itself. -
0 6 * * 1 retrain.sh— weekly retraining before Monday morning. -
A job that worked by hand and fails on schedule — almost always
PATH, or a relative path, or a virtualenv that was never activated. -
A log that has silently recorded a failure every night for three months, which is what happens when nothing alerts.
Implications: security, privacy, performance, scalability, and cost
-
Security: a scheduled job runs with your privileges, unattended, forever. It is a standing capability, and it should hold the least it needs.
-
Never put a secret in a crontab line. The table is readable and the command line is visible in
ps. Read it from a file the script owns. -
A writable script on a schedule is a persistence mechanism. If an attacker can edit it, they have a job that runs as you, on a timer.
-
Performance: overlapping runs compound. A five-minute job that takes six minutes will eventually have ten copies running.
-
Cost: an unwatched job can be expensive. A retraining job that fails at step one still spins up the machine every week.
-
Cost of getting it wrong is silence. The failure mode of automation is not a crash; it is nothing happening and nobody noticing.
Alternatives: free, open source, and commercial
| Tool | Type | Where it fits |
|---|---|---|
| cron | Free, built in | One machine, simple schedules, universal availability |
| systemd timers | Free, built in | Linux, when you want logging and dependencies |
| launchd | Free, built in | macOS, including jobs that run on events rather than times |
at | Free, built in | A single run, once, at a stated time |
| GitHub Actions schedules | Free tier | Jobs that belong with the repository |
| Airflow, Dagster, Prefect | Free, open source | Real pipelines: dependencies, retries, backfills |
-
Move off cron when you need any of three things: to know a job ran, to make one job wait for another, or to retry a failure.
-
Do not reach for Airflow on day one. A logged, idempotent cron job solves more problems than its reputation suggests.
Comparison with related concepts
| Term | What it actually is |
|---|---|
| cron | The daemon that launches jobs on a schedule |
| crontab | The table of rules, one line per job |
| Job | One schedule paired with one command |
| Idempotent | Safe to run more than once |
| Daemon | A background process with no terminal |
| Orchestrator | A scheduler that also understands dependencies |
When to use it — and when not to
-
Use a schedule when the work is genuinely periodic and does not depend on anything else finishing.
-
Use it when the machine is always on. A laptop that sleeps is a poor host for anything that matters.
-
Use
atfor a one-off, rather than adding and removing a cron line. -
Do not schedule anything you have not run by hand first, twice.
-
Do not schedule a job with no log. You are choosing not to find out.
-
Do not use cron for a pipeline. Three cron lines timed to run “long enough apart” is a race condition with a schedule attached.
The AI connection
-
Data collection runs on a schedule — scraping, API pulls, exports — long before any model exists.
-
Retraining cadence is a scheduling decision, and it is usually a cron line before it is anything more sophisticated.
-
Drift monitoring is a scheduled job that compares today’s inputs against the training distribution and complains.
-
The idempotence requirement is sharper with data. A job that appends the same rows twice does not crash; it silently corrupts your training set.
-
GPU jobs are expensive to run twice, which is why the check-before-acting habit matters more here than almost anywhere.
Knowledge check
quiz:
question: "A script runs perfectly when you execute it yourself, but produces nothing when cron runs it at the scheduled time. What is the most likely cause?"
options:
- "cron provides a minimal environment, so a command or path the script relied on is not found"
- "The crontab line has the minute and hour fields in the wrong order"
- "cron cannot execute shell scripts without the interpreter named explicitly"
- "The script needs administrator rights that cron does not grant"
answer_index: 0
explanation: "cron reads none of your shell configuration. Its PATH is short, its working directory is your home rather than your project, and no virtual environment is active — so a command that resolved fine in your terminal is simply not found."
Hands-on exercise
Schedule a job, make it fail, then find out why from its log. Worked through fully in the Day 14 lab directory.
| What you are doing | Command |
|---|---|
| See what cron sees | env -i /bin/sh -c 'echo $PATH' |
| Write a logging script | echo "[$(date)] ran" >> "$HOME/lab.log" |
| Edit your schedule | crontab -e |
| Schedule it every minute | * * * * * /full/path/job.sh >> /full/path/job.log 2>&1 |
| Watch the log fill | tail -f ~/job.log |
| List and then remove it | crontab -l then crontab -e |
Expected output
/usr/bin:/bin
[Sun Aug 30 14:03:00 2026] ran
[Sun Aug 30 14:04:00 2026] ran
/bin/sh: brew: command not found
- cron’s PATH is two directories, not the dozen your shell has.
- The failure is captured because of
2>&1— without it you would see an empty log and no explanation.
Validate your work
-
You compared cron’s
PATHwith your shell’s and can state the difference -
Your scheduled job appended timestamped lines to a log
-
You caused a failure deliberately and found it in the log rather than by guessing
-
Your script uses absolute paths throughout
-
You removed the job when finished
-
bash tests/run_tests.shreports 0 failures
Troubleshooting
-
Nothing happens at all. Check
crontab -l, and check the line ends with a newline — some cron implementations ignore a final line without one. -
command not foundin the log. Use the absolute path, or setPATHat the top of the script. -
The log is empty even though the job ran. You redirected stdout but not stderr. Add
2>&1. -
macOS refuses to let cron read your files. Grant Full Disk Access to
cron, or uselaunchdinstead. -
Percent signs vanish from your command. In a crontab,
%is special. Escape it as\%.
Common mistakes
- Testing only in your own shell, where everything resolves.
- Relative paths, which resolve against your home directory, not your project.
- No redirection, so the output goes nowhere and failures are invisible.
- Forgetting the minute field, turning one daily run into sixty.
- Scheduling before the script is safe to run twice.
Practice assignment
-
Take one script from Day 12 and make it schedulable: absolute paths, strict mode, timestamped logging, and a meaningful exit code.
-
Run it under a stripped environment with
env -ibefore scheduling it, and fix what breaks. -
Schedule it every minute for ten minutes, then read the log and remove the job.
-
Make it idempotent: run it twice in a row and prove the second run changed nothing.
-
Write down what would tell you it had stopped working, then decide whether that is good enough.
Extension challenge
-
Add a lock file so the job refuses to start if a previous run is still going. Prove it by making the job sleep longer than its own interval.
-
Convert one cron job to a systemd timer or a launchd agent, and compare what you get: logging, status, and what happens to a run missed while the machine slept.
-
Add failure alerting. Make the job send a notification only when it fails, and test that path deliberately — an alert you have never seen fire is not an alert.
-
Schedule the same job on two machines and think through what happens when both run it. That question is the beginning of distributed systems.
Quiz
Q1. In a standard crontab line, what are the five time fields in order?
- Hour, minute, day of month, month, day of week
- Minute, hour, day of week, day of month, month
- Second, minute, hour, day, month
- Minute, hour, day of month, month, day of week
Show answer
Answer: D. Minute, hour, day of month, month, day of week
A crontab line is minute, hour, day-of-month, month, day-of-week, followed by the command. Because the first field is minutes (not hours), reading the fields smallest-unit-first is the reliable way to avoid the most common cron mistake.
Q2. What does the schedule `0 9 * * 1` mean?
- At 09:00 every Monday
- Every 9 minutes on Mondays
- At 9 a.m. on the 1st of every month
- At minute 0 of every hour on the 9th
Show answer
Answer: A. At 09:00 every Monday
Minute 0, hour 9, any day of month, any month, and day-of-week 1 (Monday). The two star fields remove any calendar-date restriction, so it runs at nine o'clock sharp every Monday.
Q3. Why must a script that runs unattended on a schedule write to a log file?
- Log files make the script run faster
- At 3 a.m. nobody is watching; the log is the only record
- cron refuses to run any script that does not log
- Logging is required to convert bytes to gigabytes
Show answer
Answer: B. At 3 a.m. nobody is watching; the log is the only record
A scheduled job runs while no one is watching, so without a log a failure is silent until someone notices the missing result. The log is the unattended job's only way to report what it did and whether it succeeded.
Q4. What does it mean for a task to be idempotent?
- It runs faster each time it is repeated
- It can only ever be run a single time
- Running it twice does no more harm than running it once
- It requires administrator privileges to run
Show answer
Answer: C. Running it twice does no more harm than running it once
An idempotent job checks the current state before acting, so re-running it is harmless. This matters for scheduled jobs, which may fire twice or run on already-processed data — a non-idempotent job could double-apply and corrupt data.
Q5. What is the purpose of a dry-run mode in a script that moves or deletes files?
- It makes the file operations permanent and irreversible
- It compresses the files before moving them
- It schedules the script to run later with cron
- It prints what the script would do without changing anything
Show answer
Answer: D. It prints what the script would do without changing anything
A dry run prints the intended actions and changes nothing, letting you verify a destructive script before trusting it. Building the dry-run mode first is the habit that protects your data.
Q6. On modern macOS, which scheduler is the recommended tool, with cron deprecated though still present?
- launchd
- Windows Task Scheduler
- systemd timers
- at
Show answer
Answer: A. launchd
Apple introduced launchd in 2005 and it is now the recommended scheduler on macOS. It has the practical advantage of catching up on jobs that were missed while the Mac was asleep, which cron never runs.
Q7. A colleague says "the cron job is broken." What is usually the real cause?
- cron itself has crashed and must be reinstalled
- A bug in the script or its environment; cron only launches it
- The schedule fields were written in the wrong century
- The computer has run out of transistors
Show answer
Answer: B. A bug in the script or its environment; cron only launches it
cron only launches the command at the scheduled time; it does not understand or retry the job. Most "cron problems" are ordinary script bugs — often a relative path or missing environment variable — that reproduce when you run the command by hand.
Q8. Which tool is designed to run a command once at a single future time rather than on a repeating schedule?
- crontab
- launchd
- at
- systemd timer
Show answer
Answer: C. at
The `at` command schedules a one-off job for a single future time, such as `echo "command" | at 20:00`. cron and the other schedulers are for recurring jobs; `at` is for "just once, later."
Q9. Why should you quote variables such as `"$file"` in a scheduled shell script?
- Quoting makes cron run the job on time
- Quoting converts the file to a different format
- Unquoted variables are automatically deleted by the shell itself
- A filename with a space would break the command into pieces
Show answer
Answer: D. A filename with a space would break the command into pieces
cron runs commands through the shell, which splits unquoted values on spaces. Quoting every variable keeps a filename like "my report.pdf" as a single argument, so a space or special character cannot turn the command into something you did not intend — a safety essential for unattended jobs.
Glossary
- automation
- Pairing a script that performs a task with a scheduler that runs it at chosen times, so the work happens reliably without a person doing it by hand.
- cron
- The classic Unix scheduler: a background program that wakes every minute, checks a table of jobs against the current time, and runs the ones that match.
- crontab
- The cron table — a per-user list of scheduled jobs, one job per line; edited with crontab -e and viewed with crontab -l.
- cron expression
- The five whitespace-separated time fields at the start of a crontab line — minute, hour, day-of-month, month, day-of-week — that say when a job runs.
- scheduled task
- A command or script set to run automatically at specified times or intervals by a scheduler, rather than being started by hand.
- launchd
- The service manager and scheduler on macOS, introduced in 2005 and now the recommended tool there; it can also run jobs that were missed while the Mac was asleep.
- systemd timer
- A scheduling unit on most Linux distributions that runs a paired service on a schedule, managed and logged like any other systemd service.
- idempotent
- A property of a task that makes running it more than once no more harmful than running it once, because it checks the current state before acting.
- dry run
- A mode in which a script prints the actions it would take without performing any of them, so you can safely preview a destructive job before trusting it.
- logging
- Writing a record of what a program did — usually timestamped lines appended to a file — which is essential for unattended jobs that have no screen to watch.
- at
- A Unix command that runs a job once at a single future time rather than on a recurring schedule, for one-off delayed or reminder tasks.
- Task Scheduler
- The built-in scheduler on Microsoft Windows, used through its graphical tool or commands like schtasks to run programs at set times.
Sources and further reading
- Cron — Wikipedia (accessed 2026-07-12) — Overview of cron's history, the crontab syntax, and its variants.
- crontab(5) — man page — man7.org (accessed 2026-07-12) — The authoritative reference for the crontab file format and its five time fields.
- The Linux Command Line — William Shotts (accessed 2026-07-12) — Free book and site covering the shell, scripting, and everyday automation.
- The Missing Semester of Your CS Education — Shell Tools and Scripting — MIT (accessed 2026-07-12) — Lecture and exercises on shell scripting and automation, including cron.
- Bash Reference Manual — GNU (accessed 2026-07-12) — The official reference for Bash, the shell that runs the scripts in this lesson.
Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.