Computing Foundations › The Command Line › Day 12
Day 12: Shell Scripting: Variables, Loops, and Conditionals
After this lesson you will be able to turn a sequence of commands into a reliable shell script — using variables, quoting, command substitution, arguments, conditionals, loops, and functions under strict mode — so your work runs the same way every time and is genuinely reproducible.
Hands-on lab for this lesson
Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/computing-foundations/day-012-shell-scripting-variables-loops-and-conditionals
- Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git cd ai-roadmap-365.github.io - Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
cd labs/sections/computing-foundations/day-012-shell-scripting-variables-loops-and-conditionals - Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
- Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
bash tests/run_tests.sh # or the test command named in the lab README
You can also open the lab as a local page (works offline, shows the file tree and expected output).
Learning objectives
By the end of this lesson you will be able to:
- Save a sequence of commands as a script, add the shebang #!/usr/bin/env bash, and make it executable or run it with bash
- Assign and read variables, and explain why quoting every expansion ("$var") prevents word-splitting and globbing bugs
- Use command substitution $(...) to turn a command's output into a value, and read positional parameters $1, $@, and $# with a sensible default
- Write conditionals with if, test/[ ], and [[ ]], and explain how exit codes drive the branch, including the difference between string and integer comparisons
- Write for and while loops and package reusable steps into functions with local variables
- Add set -euo pipefail to make a script stop on errors, catch unset variables, and detect pipeline failures, and connect this discipline to reproducible AI pipelines
Prerequisites
- Days 8–11: comfort in the terminal, navigating the filesystem, working with text, and setting environment variables
- A computer running macOS or Linux (or Windows with WSL) and a text editor
Why this matters
-
Every AI project accumulates scripts — fetch the data, preprocess it, launch the run, collect the results — and they are almost always shell scripts.
-
A script is a decision written down. It runs the same way tomorrow, on a server, without you awake.
-
Everything you learned this week becomes reusable today. Pipelines, paths, variables and permissions all combine here.
-
Strict mode is the difference between a failed run that tells you and one that quietly produces a corrupt dataset.
-
You will read far more shell than you write. Every Dockerfile, CI config and install guide is made of it.
The idea in plain language
-
Anything you can type, you can save. A script is a file of commands, run top to bottom.
-
Variables hold values, and
$namereads them back. -
Conditionals branch on whether a command succeeded, not on a boolean value.
-
Loops repeat a block — once per item, or until something stops being true.
-
Functions name a useful sequence so you write it once and call it anywhere.
-
The shell is a real programming language, with one unusual property: every statement can be a program.
Historical background
-
The Thompson shell of 1971 could not script. It ran commands and little else.
-
The Bourne shell, 1979, added the language — variables, conditionals, loops, functions — and its syntax is what you will type today.
-
basharrived in 1989 as the GNU project’s free replacement, mostly compatible and considerably more capable. The name is a joke: Bourne-again shell. -
POSIX standardised the core in 1992, which is why a careful script runs unchanged on machines decades and vendors apart.
-
macOS moved its default shell to zsh in 2019 for licensing reasons, which is why your laptop and your servers may disagree — and why the shebang matters.
What it is — and what it is not
It is:
- A file of commands with a shebang line naming its interpreter.
- A real language: variables, branches, loops, functions, arguments, return codes.
- The glue layer that starts and connects other programs.
It is not:
-
Not a general-purpose programming language. It has no real data structures, its arithmetic is awkward, and its error handling is primitive.
-
Not fast at computation. Every loop iteration may launch a process.
-
Not safe by default. Unquoted variables and ignored failures are the defaults, which is why strict mode exists.
-
Not the right tool past a hundred lines or so. At that point, reach for Python.
Why it was created and what problems it solves
-
Repetition. A sequence you typed twice should be a file you run.
-
Reliability. Typing seven commands in order, correctly, at 2am, is not a plan.
-
Documentation. A script is an executable description of how something is done.
-
Composition. It is the layer that wires together programs written in every other language.
-
Automation. Anything in a script can be scheduled, triggered by a hook, or run in a container — the subject of Day 14.
How it works
From a command to a script
#!/usr/bin/env bash
set -euo pipefail
echo "Starting up"
echo "Today is $(date '+%Y-%m-%d')"
-
The shebang is
#!plus the interpreter path, and only matters when the file is run as a program. -
#!/usr/bin/env bashfinds bash onPATH, which is portable;/bin/bashis not, across macOS and Linux.
Variables and quoting
name="Ada"
greeting="Hello, ${name}"
echo "${greeting}"
- No spaces around the
=. With spaces, the shell reads a command name. - Braces mark where the name ends, so
${name}_backupis unambiguous. - Unquoted expansion does two things you rarely want: word splitting at spaces,
and globbing of
*and?.
Command substitution
today="$(date '+%Y-%m-%d')"
count="$(ls -1 | wc -l)"
-
$(...)runs the command and substitutes what it printed, trailing newline trimmed. -
This is how a script reacts to reality rather than hard-coding an answer.
-
Prefer
$(...)to backticks. It nests cleanly and reads better.
Reading arguments
target="${1:-.}"
echo "Working on: ${target}, with $# argument(s)"
$1,$2are the positional parameters;$#is how many there were."$@"is all of them, each kept intact — quote it, always.${1:-default}supplies a fallback, which makes a script both flexible and forgiving.
Conditionals
if [[ -d "${target}" ]]; then
echo "It is a directory"
else
echo "No such directory: ${target}"
fi
-
Prefer
[[ ... ]]in bash. It does not word-split its operands and supports pattern matching. -
Use
[ ... ]when the script must run under plain POSIXsh.
Loops and functions
for f in *.txt; do
echo "Found: ${f}"
done
while read -r line; do
echo "Line: ${line}"
done < notes.txt
log() {
local msg="$*"
echo "[$(date '+%H:%M:%S')] ${msg}"
}
read -rstops it mangling backslashes.- Inside a function,
$1is the function’s argument, not the script’s. - Declare working variables
localso they do not leak into the rest of the script.
Strict mode: set -euo pipefail
-
A default shell is too forgiving to trust. One line near the top of every serious script fixes that, and it belongs there before anything else.
-
-estops at the first failing command, instead of blundering onward through steps whose inputs never arrived. -
-uturns a mistyped${targett}from a silent empty string into an immediate, loud error — which is the single most valuable of the three. -
-o pipefailreports a pipeline as failed when any stage failed, not only the last one. -
Adopt the line as a reflex. Typing it should cost you no thought at all.
Asking a question
read -r -p "Delete these files? [y/N] " answer
[[ "${answer}" == "y" ]] && echo "Deleting..." || echo "Cancelled"
-pprints a prompt. This is how interactive scripts confirm dangerous actions.- Automated pipelines pass arguments instead, so the script can run unattended.
An everyday analogy
A recipe card.
| The kitchen | The script |
|---|---|
| The recipe card | The script file |
| ”Serves 4” at the top | The shebang and strict mode |
| Ingredients list | Variables |
| ”If the sauce is thin, simmer” | A conditional |
| ”Repeat for each portion” | A loop |
| ”Make the roux (see page 12)“ | A function |
| Taste before serving | Checking exit codes |
-
A recipe you follow twice should be written down, or you will get it slightly different each time.
-
A good recipe says what to do when something goes wrong. A script without strict mode is a recipe that ignores a burnt pan and carries on.
Examples in practice
-
A backup script that names its output with
$(date)so runs never overwrite each other. -
A preprocessing loop over every file in a directory, skipping ones already done.
-
A launcher that reads
$1as the config file and defaults toconfig.yml. -
A retry loop that waits until a service answers before starting the real work.
-
A cleanup function that runs on exit via
trap, so temporary files go away even when the script fails.
Implications: security, privacy, performance, scalability, and cost
-
Security: unquoted variables are an injection vector. A filename containing a space or a semicolon can change what your script does.
-
Never build a command by string concatenation from untrusted input. The shell will happily execute what you accidentally assembled.
-
Secrets belong in the environment, not in the script and not in its arguments —
psshows arguments to everyone. -
Performance: a loop that launches a process per item is slow. Ten thousand iterations means ten thousand process creations. Prefer one pipeline over a loop where you can.
-
Scalability: shell scales badly past a few hundred lines. No types, no modules, no test framework worth the name.
-
Cost: a script without
set -ecan burn hours of compute producing output from a step that already failed.
Alternatives: free, open source, and commercial
| Tool | Type | When to use it instead |
|---|---|---|
| bash | Free, built in | Glue, automation, anything under ~100 lines |
POSIX sh | Free, built in | Maximum portability, minimum features |
| Python | Free, open source | Data structures, real error handling, tests |
make | Free, built in | Dependency graphs — rebuild only what changed |
just | Free, open source | A friendlier task runner, without make’s traps |
shellcheck | Free, open source | Not an alternative — a linter you should always run |
-
shellcheckis not optional. It catches unquoted variables, uselesscat, and a dozen subtle traps, and it takes a second. -
The switch to Python comes when you need a dictionary, a real test, or an exception. Those three are the signal.
Comparison with related concepts
| Term | What it actually is |
|---|---|
| Script | A file of commands for an interpreter |
| Shebang | The first line naming that interpreter |
| Exit code | The integer a command returns; 0 is success |
| Positional parameter | $1, $2 — an argument by position |
| Command substitution | $(...) — a command’s output used as a value |
| Strict mode | set -euo pipefail, the safety line |
When to use it — and when not to
-
Use shell when the job is running other programs in order, with a little logic between them.
-
Use it when the script must run anywhere with nothing installed.
-
Use it for anything you have typed twice.
-
Switch to Python when you need a data structure, real error handling, or tests.
-
Switch when the script passes a hundred lines, or when you find yourself parsing anything structured.
-
Do not write shell that generates shell. That is the point where the next person cannot debug it, and the next person is you.
The AI connection
-
Every training run starts from a script: set the environment, check the data exists, launch, capture the log.
-
Dataset preparation is loops and pipelines long before it is Python.
-
Container entrypoints are shell scripts, and a missing
set -ethere means a container that reports success while doing nothing. -
Sweep launchers are
forloops over hyperparameter values, which is the simplest experiment framework there is and often enough. -
Reproducibility means the commands are written down. A script is the cheapest form of that promise.
Knowledge check
quiz:
question: "A script runs `download_data | unpack` and reports success, but the output directory is empty. Which strict-mode option would have caught this?"
options:
- "-o pipefail, because a pipeline's exit code otherwise comes only from its last command"
- "-e, because the script would have exited at the first failing command"
- "-u, because the output path variable was never set"
- "-x, because tracing would have shown which command failed"
answer_index: 0
explanation: "Without pipefail, a pipeline reports whatever its final command returned. unpack succeeded at unpacking nothing, so the pipeline looked fine even though the download had already failed."
Hands-on exercise
Write a script that takes an argument, loops over files, and fails loudly. Worked through fully in the Day 12 lab directory.
| What you are doing | Command or line |
|---|---|
| Start the file properly | #!/usr/bin/env bash then set -euo pipefail |
| Take an argument with a default | target="${1:-.}" |
| Guard against a bad input | [[ -d "$target" ]] || { echo "no such dir" >&2; exit 1; } |
| Loop over matching files | for f in "$target"/*.txt; do ... done |
| Capture a value | count="$(ls -1 "$target" | wc -l)" |
| Run it | chmod +x tool.sh then ./tool.sh reports |
| Lint it | shellcheck tool.sh |
Expected output
[14:02:11] Working on: reports
[14:02:11] Found: reports/summary.txt
[14:02:11] 3 entries
./tool.sh: line 9: nosuchcmd: command not found
- The script stops at the failing line rather than continuing — that is
-e. - The error went to stderr, so it stays out of any pipeline reading the output.
Validate your work
- Your script has a shebang and a strict-mode line
- It accepts an argument and defaults sensibly when given none
- It exits non-zero with a message on stderr for a bad input
- Every variable expansion is quoted
-
shellcheckreports no warnings -
bash tests/run_tests.shreports 0 failures
Troubleshooting
-
command not foundon your own script. Use./tool.sh; the current directory is not onPATH. -
Permission denied.chmod +x, or run it asbash tool.sh. -
unbound variableafter addingset -u. That is the flag working. Find the typo it caught. -
A loop over
*.txtruns once with a literal*.txt. Nothing matched, and the pattern was passed through unchanged. Guard with a test. -
[: too many arguments. An unquoted variable split into several words.
Common mistakes
- Spaces around
=in an assignment. - Comparing numbers with
=or strings with-eq. - Forgetting quotes, which works until the first filename with a space.
- Omitting
set -euo pipefailand then trusting the output. - Writing errors to stdout, where they contaminate whatever reads the script.
Practice assignment
-
Convert three commands you typed this week into a script with an argument and a default.
-
Add a
trapthat cleans up a temporary directory on exit, and prove it runs even when the script fails halfway. -
Break your script deliberately in three ways — a typo’d variable, a failing command mid-pipeline, an unquoted filename with a space — and confirm strict mode catches each.
-
Run
shellcheckon every script you have ever written and fix what it finds.
Extension challenge
-
Write a script that retries with backoff until a URL responds, giving up after five attempts, and returns a meaningful exit code either way.
-
Measure the cost of process creation. Loop ten thousand times calling an external command, then do the same work with one pipeline. The ratio is why the “prefer a pipeline” advice exists.
-
Add argument parsing that supports both
-vand--verbose, and a--helpthat exits 0. -
Make one of your scripts pass
shellcheckat its strictest setting, and note which warnings you had to think hardest about.
Quiz
Q1. What is the purpose of the first line `#!/usr/bin/env bash` in a script?
- It is a comment the shell ignores completely
- It imports a library of extra shell commands
- It is the shebang: run this file with bash
- It makes the file executable automatically
Show answer
Answer: C. It is the shebang: run this file with bash
The shebang `#!` followed by an interpreter path tells the system which program should run the file. Using `/usr/bin/env bash` finds bash on the user's PATH, which is more portable than a hard-coded path. It does not, by itself, set the executable permission bit.
Q2. Why should you almost always quote a variable expansion, writing `rm "$file"` instead of `rm $file`?
- Quoting makes the script run faster
- Quotes convert the value to uppercase
- Unquoted variables are a syntax error in bash
- Unquoted values are word-split and globbed
Show answer
Answer: D. Unquoted values are word-split and globbed
Without quotes the shell splits the value at spaces and expands glob characters like `*`. A value such as `my notes.txt` becomes two arguments, and `*` becomes every file in the directory. Quoting passes the value through as a single, literal argument.
Q3. What does `set -euo pipefail` do?
- Exit on error, on unset variables, and on pipeline failure
- Turns on verbose logging of every command
- Disables all error checking so the script never stops
- Sets three environment variables named e, u, and o
Show answer
Answer: A. Exit on error, on unset variables, and on pipeline failure
`-e` exits on the first command that fails, `-u` treats use of an unset variable as an error (catching typos), and `-o pipefail` makes a pipeline report failure if any command in it fails, not just the last. Together they turn a forgiving shell into a strict one that stops when something is wrong.
Q4. What does command substitution `$(date +%Y)` produce?
- The literal text "date +%Y"
- The output of `date +%Y`, substituted in place
- An error, because commands cannot appear inside scripts
- The exit code of the date command
Show answer
Answer: B. The output of `date +%Y`, substituted in place
Command substitution runs the command inside `$(...)`, captures what it prints, trims the trailing newline, and substitutes the result as a value. It lets a script react to real information rather than a hard-coded guess.
Q5. A script uses `if [ $count = 5 ]` to compare a number and behaves oddly. What is the correct integer comparison?
- `if [ $count == 5 ]`
- `if [ $count -is 5 ]`
- `if [ $count -eq 5 ]`
- `if ( count = 5 )`
Show answer
Answer: C. `if [ $count -eq 5 ]`
In test expressions, `=` compares strings while `-eq`, `-gt`, `-lt`, and similar operators compare integers. Use `[ "$count" -eq 5 ]` for numbers. Quoting the variable also avoids an error when the value is empty.
Q6. Which loop is the natural choice for reading a file one line at a time?
- A `for` loop over `*.txt`
- Recursion, since shells have no loops
- A `case` statement
- A `while read -r line` loop
Show answer
Answer: D. A `while read -r line` loop
A `while read -r line` loop reading from a redirected file processes one line per iteration and stops at end of file, which is exactly the "repeat while the command succeeds" pattern `while` is built for. A `for` loop suits iterating over a known list such as files matching a pattern.
Q7. What is the difference between `$1` and `$@` in a script?
- `$1` is the first argument; `$@` is all of them
- They are identical
- `$1` is the exit code; `$@` is the script name
- `$1` is the number of arguments; `$@` is the first one
Show answer
Answer: A. `$1` is the first argument; `$@` is all of them
`$1` is the first argument passed to the script, `$2` the second, and so on. `$@` expands to all the arguments (quote it as `"$@"` to keep each one intact), and `$#` is the count of arguments.
Q8. Why is `[[ ... ]]` generally preferred over `[ ... ]` in a bash script?
- It runs on every shell including minimal POSIX sh
- It is a bash built-in that does not word-split and supports patterns
- It is required for the shebang to work
- It automatically quotes every variable for you regardless of shell
Show answer
Answer: B. It is a bash built-in that does not word-split and supports patterns
`[[ ]]` is a bash keyword that avoids the word-splitting and globbing pitfalls of the older `[` command and adds pattern matching and logical operators inside the brackets. Use `[ ]` only when a script must run under plain POSIX `sh`, which lacks `[[ ]]`.
Glossary
- shebang
- The `#!` characters on a script's first line, followed by the path to an interpreter (e.g. `#!/usr/bin/env bash`), telling the system which program should run the file.
- variable
- A named holder for a value in a script, assigned with `name=value` (no spaces around `=`) and read with `$name` or `${name}`.
- quoting
- Wrapping a value in double quotes, as in `"$file"`, to stop the shell from word-splitting it at spaces and expanding glob characters like `*` — the single most important habit for correct scripts.
- command substitution
- The `$(command)` form, which runs a command and replaces itself with that command's captured output, letting a script use real results as values.
- exit code
- The integer a command returns when it finishes — `0` for success and non-zero for failure — which conditionals and loops branch on.
- conditional
- An `if`/`then`/`else` construct that runs commands based on whether a test succeeds, where the test is any command judged by its exit code.
- loop
- A construct that repeats a block of commands: `for` iterates over a known list, while `while` (and `until`) repeats based on a command's exit code.
- function
- A named, reusable group of commands defined with `name() { ... }` and called like any command; inside it, `$1` and `$2` are the arguments passed to the function.
- positional parameter
- The numbered inputs a script receives on the command line — `$1`, `$2`, and so on — with `$#` giving their count and `$@` expanding to all of them.
- test command
- The `[ ... ]` (also spelled `test`) or the bash-only `[[ ... ]]` construct that evaluates a condition, such as `-d` for "is a directory", and returns an exit code.
- ShellCheck
- A free, open-source static analyzer (linter) for shell scripts that flags bugs like unquoted variables and misused test brackets before the script is ever run.
- set -e
- A shell option (part of `set -euo pipefail`) that makes a script exit immediately when any command fails, instead of blindly continuing after an error.
- strict mode
- The convention of putting `set -euo pipefail` near the top of a script so it stops on errors, errors on unset variables, and detects failures anywhere in a pipeline.
Sources and further reading
- Bash Reference Manual — GNU (accessed 2026-07-12)
- The Linux Command Line — William Shotts (accessed 2026-07-12)
- Google Shell Style Guide — Google (accessed 2026-07-12)
- The Missing Semester of Your CS Education — Shell Tools and Scripting — MIT (accessed 2026-07-12)
- Shell script — Wikipedia (accessed 2026-07-12)
Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.