Programming with Python › Python for Automation and the Web › Day 80
Hands-on lab — Day 80: Building CLIs with argparse
- ← Back to the Day 80 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/programming-with-python/day-080-building-clis-with-argparse/
Commands
Setup
cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
python3 --version
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt Run
python3 examples/by_hand.py
python3 examples/notes.py --help
python3 examples/notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
echo 'from a pipe' | python3 examples/notes.py add - --tag inbox --on 2026-03-03 --store notes.json
python3 examples/notes.py list --format json --store notes.json
python3 examples/notes.py remove 2 --dry-run --store notes.json
python3 starter/notes.py --help Test
bash tests/run_tests.sh File tree
examples/by_hand.py examples/notes.py expected-output/FIELDS.md expected-output/help-output.txt expected-output/sample-run.txt expected-output/test-run.txt metadata.yml README.md requirements/README.md requirements/requirements.txt security.md starter/notes.py tests/conftest.py tests/run_tests.sh tests/test_parser.py troubleshooting.md
Lab README
Day 080 lab — A Tool You Would Actually Install
Lesson
- Lesson title: Building CLIs with argparse
- Day number: 80 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-080-building-clis-with-argparse
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-080-building-clis-with-argparsewhen the site is running.
Purpose
On Day 56 you built a command-line tool by reading sys.argv yourself. It
worked, and for what it did it was the right choice. This lab starts by
showing you, in eight ordinary command lines, exactly where that approach
stops working — and then builds the thing that replaces it.
You will finish with notes: a real multi-subcommand tool with add, list,
search, export and remove, over a small JSON store. Not a toy. It has
the things users expect and notice the absence of — -h, --help,
--version, short and long option forms, -- to end option parsing, tab-free
help text organised into groups, sensible exit codes — and the things that
separate a tool from a script: values converted and validated at the parser
rather than three functions later, --dry-run on the one destructive command,
results on standard output and diagnostics on standard error so it composes in
a pipeline, a - argument that reads the note from standard input, and
configuration that follows the precedence users already expect (flag, then
environment, then default).
The single most useful technique here is subcommands with
add_subparsers and set_defaults(func=...). It is how git, docker, pip,
and every other tool with more than one verb is structured, and it removes
every if command == "add" from your program.
The test suite is unusual and worth reading before you run it: most of it launches your program as a subprocess and inspects what a shell can see — the exit code, standard output, and standard error, captured to separate files. That is the only honest way to test a command-line interface, and it is why one check can prove your streams are genuinely separated rather than merely looking right in a terminal.
Learning objectives
- Recognise the point at which hand-parsing
sys.argvstarts producing wrong answers silently, using a working demonstration rather than an assertion. - Build an
argparse.ArgumentParserwithprog,description,epilogand per-argumenthelp, and treat the resulting--helpoutput as a deliverable rather than a side effect. - Use
add_subparserswithset_defaults(func=...)so dispatch isargs.func(args, streams)and no branch on the command name exists anywhere. - Write custom
type=callables that convert and validate, raisingargparse.ArgumentTypeErrorso a bad value becomes a usage message and exit 2 instead of a traceback. - Use
choices,default,nargs="+",action="store_true",action="append", argument groups, and a mutually exclusive group, and say what each one buys. - Separate results (standard output) from diagnostics (standard error), and prove the separation with a test that captures the two streams apart.
- Read a note from standard input when the argument is
-, and detect a terminal so the tool explains itself instead of hanging. - Implement
--dry-runas a real guarantee — validated for real, written never — and verify it by hashing the store before and after. - Resolve configuration by the expected precedence: flag, then environment, then built-in default.
- Make the parser testable by having
parse_args(argv)take an explicit list, and test both in-process and as a subprocess.
Prerequisites
- The Day 80 lesson.
- Day 56: building a data-driven CLI with
sys.argv— this lab is its sequel. - Days 8–14: the command line itself. Pipes, redirection, and exit codes are assumed knowledge here.
- Days 64–66: reading and writing files, JSON, and exception strategy.
- Days 71–74: pytest, and Day 74's argument about injecting a boundary rather
than reaching for it —
parse_args(argv)and theStreamsobject are that idea applied to the command line. - Day 69: type hints on public functions.
- A text editor, a terminal, and Python 3.
Supported operating systems
- macOS — fully supported (tested on macOS 26.5.1, Apple Silicon, Python 3.14.0, pytest 9.1.1, bash 3.2.57).
- Linux — fully supported (any distribution with Python 3 and bash).
- Windows — use WSL and follow the Linux path. On a native Windows console
substitute
pythonforpython3, and note that the suite's banner contains an em dash, so a UTF-8 terminal is needed for it to render. Exit codes, stream separation, and the JSON bytes are identical everywhere.
Hardware requirements
Any computer that runs Python 3. The store this lab writes is a few hundred bytes; the whole suite finishes in about a second. No special memory, disk, GPU, or network.
Required software
python3(3.8 or newer; tested on 3.14.0). The tool itself uses only the standard library:argparse,json,csv,os,sys,pathlib,datetime,dataclasses.pytestfor the in-process half of the suite — the only dependency, and only for the tests. Seerequirements/README.md.bashfor the test runner (preinstalled on macOS and Linux).
Free and open-source options
Everything here is free and open source: Python and its standard library, bash, and pytest (MIT licence). No account, API key, network access, or purchase is needed at any point.
The lesson's Alternatives section covers the other options for this job —
click and typer, both free and open source, and docopt, which is free
but much less actively maintained. None of them is required here, and the
reason argparse is the right teaching choice is precisely that it needs no
install at all.
Installation
cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
python3 --version # confirm Python 3.8+
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
If you already have pytest somewhere, you can skip the virtual environment and
point the runner at it: PYTEST=/path/to/pytest bash tests/run_tests.sh.
File structure
day-080-building-clis-with-argparse/
├── README.md ← you are here
├── metadata.yml ← machine-readable lab metadata
├── examples/
│ ├── by_hand.py ← the sys.argv parser, and the 4 of 8 cases it gets wrong
│ └── notes.py ← the complete reference tool
├── starter/
│ └── notes.py ← YOUR working file: eight numbered exercises
├── tests/
│ ├── run_tests.sh ← subprocess checks: exit codes and both streams
│ ├── conftest.py ← puts examples/ or starter/ on sys.path
│ └── test_parser.py ← in-process checks: parse_args with an explicit list
├── expected-output/
│ ├── sample-run.txt ← a real captured session
│ ├── help-output.txt ← every help screen the tool produces
│ ├── test-run.txt ← a real captured run of the suite
│ └── FIELDS.md ← the exact contract your tool must satisfy
├── requirements/
│ ├── requirements.txt ← pytest==9.1.1, for the tests only
│ └── README.md ← what each dependency is for
├── troubleshooting.md
└── security.md
Running the tool writes notes.json into whatever directory you run it from,
unless you say otherwise. It is safe to delete at any time.
How to run
From this directory:
## 1. See why argparse exists. Eight ordinary command lines, four wrong answers.
python3 examples/by_hand.py
## 2. Read the finished tool's user manual — which it generates itself.
python3 examples/notes.py --help
python3 examples/notes.py add --help
## 3. Drive the reference. Note that --on is explicit, so the output is stable.
python3 examples/notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
python3 examples/notes.py add 'argparse turns a script into a tool' -t python -t writing --on 2026-03-02 --store notes.json
## 4. Read a note from a pipe, which is what the '-' convention is for.
echo 'from a pipe' | python3 examples/notes.py add - --tag inbox --on 2026-03-03 --store notes.json
## 5. Results on standard output, so the tool composes.
python3 examples/notes.py list --store notes.json
python3 examples/notes.py list --format json --store notes.json | python3 -m json.tool
## 6. The dry run, and the proof that it wrote nothing.
shasum -a 256 notes.json
python3 examples/notes.py remove 2 --dry-run --store notes.json
shasum -a 256 notes.json
## 7. Three failures, three different exit codes. Check each with `echo $?`.
python3 examples/notes.py add 'x' --on 2026-13-01 --store notes.json ; echo "exit $?"
python3 examples/notes.py frobnicate ; echo "exit $?"
python3 examples/notes.py remove 99 --store notes.json ; echo "exit $?"
## 8. The stream separation, made visible.
python3 examples/notes.py list --format json -v --store notes.json > result.json 2> chatter.txt
cat result.json
cat chatter.txt
## 9. Your task: exercises 1-8 in starter/notes.py.
python3 starter/notes.py --help
## 10. Check your work.
bash tests/run_tests.sh
What the commands do
python3 examples/by_hand.py— runs a hand-rolledsys.argvparser against eight command lines a user might really type. Three work, one works by luck, and four are handled wrongly:--tag=shoppingsilently loses its value, the short form-tis swallowed as the note text, the typo--drynrunis accepted and ignored, and a missing option value raisesIndexError. This is the argument for argparse, run rather than asserted.python3 examples/notes.py --help— prints the tool's manual. Every word of it comes from the sameadd_argumentcalls that do the parsing, which is the reason it cannot drift out of date.- The
addcommands — store two notes with tags and explicit dates.--onis given explicitly so that every capture inexpected-output/is reproducible; leave it off and the note is filed under today. echo 'from a pipe' | ... add -— the-convention: read the value from standard input instead of the command line. This is how a tool joins a pipeline, and it is also how you keep a secret out of your shell history.list --format json | python3 -m json.tool— proves the result on standard output is machine-readable, with nothing else mixed into it.- The two
shasumcalls aroundremove --dry-run— print the same hash. That is what makes--dry-runa promise rather than a claim. - The three failing commands — exit 2 (a usage error argparse caught), 2 again (an unknown subcommand), and 1 (a refusal the tool itself understood). Different numbers because a script downstream needs to tell them apart.
- The redirect in step 8 —
result.jsonholds valid JSON;chatter.txtholds2 note(s) from notes.json. Two streams, two destinations, no interference. python3 starter/notes.py --help— fails until exercise 4 is done. That is the point: nothing works before the parser exists.bash tests/run_tests.sh— 76 checks while the starter is unfinished, 133 once all eight exercises are complete. Exits 0 only if every check passes.
Expected output
See expected-output/sample-run.txt for the
full captured session and
expected-output/help-output.txt for every
help screen. The heart of it:
$ python3 notes.py list --store notes.json
ID DATE TAGS TEXT
------------------------------------
1 2026-03-01 health ring the dentist
2 2026-03-02 python,writing argparse turns a script into a tool
3 2026-03-03 inbox from a pipe
exit: 0
$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json
$ python3 notes.py remove 2 --dry-run --store notes.json
would remove note 2
dry run: 1 note(s) would be removed; notes.json was not touched
exit: 0
$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json
$ python3 notes.py add 'x' --on 2026-13-01 --store notes.json
usage: notes add [-h] [--store PATH] [-v | -q] [-t TAG] [--on YYYY-MM-DD]
[--dry-run]
text
notes add: error: argument --on: '2026-13-01' is not a date in YYYY-MM-DD form (for example 2026-03-01)
exit: 2
Every date is supplied explicitly with --on and nothing reads the network or
a random number, so these bytes are reproducible on any machine with Python 3.
expected-output/FIELDS.md states the full
contract: every exit code, which stream each kind of output belongs on, the
dry-run guarantee, the precedence rules, and the in-process parser behaviour.
Validation steps
python3 examples/by_hand.pyends with8 ordinary command lines, 4 of them handled wrongly.python3 examples/notes.py --helpexits 0 and names all five subcommands.python3 examples/notes.py frobnicate; echo $?prints2, and the message appears on standard error — confirm withpython3 examples/notes.py frobnicate 2>/dev/null, which prints nothing.python3 examples/notes.py add 'x' --on 2026-13-01 >out.txt 2>err.txtleavesout.txtempty anderr.txtnon-empty. Check both. This is the single check that proves the streams were separated on purpose.- The two
shasum -a 256 notes.jsoncalls aroundremove --dry-runprint the identical hash; the one after a realremovediffers. python3 examples/notes.py search kangaroo; echo $?prints1and no output — grep's convention, not an error.echo 'x' | python3 examples/notes.py add - --on 2026-03-09 --store t.jsonexits 0 and the note text really isx.- Every exercise in
starter/notes.pyis done —grep -c 'raise NotImplementedError' starter/notes.pyreturns0. bash tests/run_tests.shreports0 failure(s).and exits 0.
Tests
bash tests/run_tests.sh
Expected final line while the starter is unfinished: 76 checks, 0 failure(s).
Once all eight exercises are complete the same battery runs a second time
against your file, giving 133 checks, 0 failure(s). Both are correct; only
the failure count matters. The command exits 0 on success and non-zero on any
failure, so it can run in CI. A full captured run is in
expected-output/test-run.txt.
The suite has two halves, and the split is the lesson:
- Subprocess checks launch the real program and inspect exit codes and the two streams separately. Only a subprocess can observe an exit code, and only separate capture can prove stream separation.
- In-process pytest checks (
tests/test_parser.py) callparse_args(["add", "hello"])andmain(argv, streams)directly, withio.StringIOstanding in for the terminal. They are far faster, give real tracebacks, and can assert on the parsed namespace — which a subprocess can never see. They are only possible becauseparse_argstakes an explicit list andmaintakes its streams as a parameter.
Two checks deserve a look before you run them. The dry-run check is paired
with a control that runs the same command without --dry-run and demands
the hash changes — without it, "the file did not change" would also be
satisfied by a tool that never writes at all. And the suite breaks a copy of
the reference with sed, then confirms the help check notices, so you know the
checks are not vacuous.
Cleanup
The lab writes only notes.json, into whatever directory you ran the tool
from, plus result.json and chatter.txt if you followed step 8:
rm -f notes.json result.json chatter.txt t.json
To reset your work: git checkout -- starter/. To remove the virtual
environment: rm -rf .venv. The test runner makes its own directory with
mktemp -d and removes it in a trap, so even a failed run leaves nothing
behind.
Troubleshooting
See troubleshooting.md for the full list: the
NotImplementedError from each unfinished exercise, the
AttributeError: 'Namespace' object has no attribute 'func' that means a
missing set_defaults or a missing required=True, the conflicting option string: -h that means a parent parser without add_help=False, why
type=iso_date() with parentheses is wrong, why a traceback means you raised
ValueError instead of ArgumentTypeError, why notes add - hangs before
exercise 7 is done, why dry-run output can arrive out of order without a
flush, and how to point the suite at an existing pytest.
Security notes
See security.md. Short version: the command line is a trust
boundary, and every type= callable is a gate on it — a value that is
converted and validated at the parser can never exist in a dangerous shape
further in. Never build a shell command by string-formatting user input; pass
a list and no shell ever sees it. Secrets do not belong in arguments, because
arguments are visible in the process list and in shell history — which is the
second reason the - stdin convention exists. json.loads cannot execute
code; pickle can. And a --dry-run that has never been tested is a claim,
not a guarantee.
Extension exercises
- Add an
editsubcommand. It should take an id and either new text as an argument or-to read from standard input. Count how many places you had to touch. If it was more than oneadd_parserblock and one handler, something in your design is not carrying its weight. - Add a config file. Read defaults from
~/.notesrc(a small JSON file) and slot it into the precedence chain between the built-in defaults and the environment: defaults, then config file, then$NOTES_STORE, then the flag. Useparser.set_defaults(**from_config)— argparse supports exactly this, and doing it any other way means writing the precedence logic by hand. - Add shell completion, honestly. argparse has no built-in completion.
Write a small bash completion function by hand that offers the five
subcommand names, and note in a comment which third-party package
(
argcomplete) would generate it for you and what that would cost in dependencies. Deciding not to add a dependency is a real engineering act. - Make the destructive path safer. Add an interactive confirmation to
remove, plus--yesto skip it. Then work out how to test the prompt — the answer is thatStreamsalready gives you the seam. - Break the streams on purpose. Change one
streams.stderr.writeincmd_removetostreams.stdout.writeand run the suite. Watch exactly one check fail and read its name. Then put it back. Knowing which check catches a mistake is worth more than being told the rule.
Navigation
- Previous day: Day 79 — Web Scraping Basics
(
labs/sections/programming-with-python/day-079-web-scraping-basics/). - Next day: Day 81 — Scheduling and Background Jobs
(
labs/sections/programming-with-python/day-081-scheduling-and-background-jobs/), which schedules a tool like this one and needs its exit codes to be right. - Week 12 project: the Personal Automation Toolkit
(
labs/sections/programming-with-python/projects/week-12/). Thenotestool is a direct rehearsal for its command-line front end.
Expected output
FIELDS.md
# Expected output — Day 080 lab
These are real captured runs from the authoring machine (macOS 26.5.1, Apple
Silicon, Python 3.14.0, pytest 9.1.1, bash 3.2.57, 2026-07-19). Every date the
tool records comes from an explicit `--on`, and nothing here reads the
network or a random number, so the same commands produce the same bytes on
any machine with Python 3.
## Files
- `sample-run.txt` — a full session with the reference tool: adding notes by
argument and through a pipe, listing as a table and as JSON, searching,
exporting CSV, a `--dry-run` with the store's SHA-256 hash printed before
and after, the real removal, the two output streams redirected to separate
files, and five different failures with their exit codes.
- `help-output.txt` — every help screen the tool produces: the root `--help`,
all five subcommand `--help` screens, and `--version`. Read this one as a
document, not as output: it is the tool's entire user manual, generated
from the same declarations that do the parsing.
- `test-run.txt` — a full run of `bash tests/run_tests.sh` with the starter
exercises unfinished: 76 checks, 0 failures, exit 0. One line in it varies
between runs — the check that reports `42 passed in 0.07s` quotes pytest's
own timing, which is a measurement and will differ by a few hundredths of a
second on your machine. Every other line is byte-stable.
## The contract your finished tool must satisfy
### Exit codes
| Command | Exit code | Why |
| --- | --- | --- |
| `notes --help`, `notes add --help`, `notes --version` | 0 | asking for help is not an error |
| a command that did its job | 0 | the only success code |
| `notes search kangaroo` with no match | 1 | grep's convention: nothing found, nothing wrong |
| `notes remove 99` when 99 does not exist | 1 | a refusal the tool understood |
| a store that is not valid JSON | 1 | a refusal, not a traceback |
| `notes` with no subcommand | 2 | usage error |
| `notes frobnicate` | 2 | usage error |
| `notes add x --on 2026-13-01` | 2 | the custom `type=` rejected the value |
| `notes list --format yaml` | 2 | outside `choices=` |
| `notes list -v -q` | 2 | mutually exclusive options |
| `notes add x --tagg typo` | 2 | an unknown option is refused, never ignored |
### Which stream each thing goes to
| Output | Stream | Reason |
| --- | --- | --- |
| the note ids from `remove --dry-run` | standard output | it is a RESULT; a person can pipe it |
| `list`, `search`, `export` output | standard output | the result again |
| `added note 3`, `removed note 2` | standard output | a short confirmation the user asked for |
| every `-v` explanation | standard error | a diagnostic; it must not pollute a pipe |
| the `dry run: ... was not touched` summary | standard error | a diagnostic |
| every error message | standard error | so `2>/dev/null` still leaves the result usable |
| argparse's own usage errors | standard error | argparse does this for you, and it is right |
The single check that proves this was implemented rather than approximated:
`notes add x --on 2026-13-01 >out.txt 2>err.txt` must leave `out.txt` **empty**
and `err.txt` **non-empty**. A program that printed its errors with a bare
`print()` passes a `2>&1` check and fails this one.
### The dry-run promise
| Step | Required |
| --- | --- |
| `shasum -a 256 notes.json` before | some hash H |
| `notes remove 2 --dry-run` | exits 0, prints `would remove note 2` on stdout |
| `shasum -a 256 notes.json` after | **exactly H** |
| `notes remove 99 --dry-run` | still exits 1 — a dry run validates for real |
| `notes remove 2` (no `--dry-run`) | exits 0, and the hash **changes** |
That last row is the control. Without it, "the file did not change" would also
be satisfied by a tool that never writes anything.
### Configuration precedence
| Command | Store used |
| --- | --- |
| `notes add x --store a.json` with `NOTES_STORE=b.json` set | `a.json` — the flag wins |
| `notes add x` with `NOTES_STORE=b.json` set | `b.json` — the environment |
| `notes add x` with nothing set | `./notes.json` — the built-in default |
### Parser behaviour, in-process
| Call | Result |
| --- | --- |
| `parse_args(["add", "hello"])` | `args.command == "add"`, `args.text == "hello"` |
| `parse_args(["remove", "3", "--dry-run"])` | `args.dry_run is True`, `args.ids == [3]` |
| `parse_args(["remove", "2", "5", "9"])` | `args.ids == [2, 5, 9]` — `nargs="+"` |
| `parse_args(["add", "x", "-t", "a", "--tag", "b"])` | `args.tag == ["a", "b"]` — `action="append"` |
| `parse_args(["add", "x", "--on", "2026-03-01"])` | `args.on` is a `datetime.date`, already converted |
| `parse_args(["add", "--", "--not-a-flag"])` | `args.text == "--not-a-flag"` |
| `parse_args(["list"]).store` | `None` — not `"notes.json"`; see `store_path()` |
| `parse_args(["list"]).func` | `is cmd_list` — the `set_defaults` dispatch |
| any usage error | raises `SystemExit` with `code == 2` |
| `parse_args(["--help"])` | raises `SystemExit` with `code == 0` |
| `iso_date("2026-13-01")` | raises `argparse.ArgumentTypeError` naming the value |
| `positive_int("0")` | raises `argparse.ArgumentTypeError` saying "at least 1" |
## Platform notes
- macOS and Linux produce identical bytes. The only thing that differs
between the two in `sample-run.txt` is the temporary directory the capture
ran in, and that never appears in the output.
- `shasum -a 256` is the macOS spelling. On most Linux distributions the
command is `sha256sum`; the hash is the same either way. The test suite
does not depend on either — it hashes the file with Python's `hashlib`, so
it works everywhere.
- **Colour.** From Python 3.13 argparse colourizes its help and usage output
when it detects a terminal. Every capture in this directory was made with
output redirected to a file, so no colour codes appear in them. Run the same
commands interactively on a recent Python and the *text* is identical while
the terminal shows it in colour. Nothing in the test suite depends on
colour, because everything it inspects is captured to a file.
- On Windows, run everything inside WSL. Two things genuinely differ on a
native Windows console: `python3` may be spelled `python`, and the em dash
in the suite's banner needs a UTF-8 terminal. Exit codes, stream
separation and the JSON bytes are unaffected.
- The suite writes only inside a directory made with `mktemp -d` and removes
it in a `trap`, so even a failed run leaves nothing behind.
help-output.txt
# Every help screen this tool produces, captured from real runs.
# Help output is the documentation most users will ever read, so it is
# worth looking at all of it in one place.
$ python3 notes.py --help
usage: notes [-h] [--version] <command> ...
Keep short notes in a JSON file you can read with your own eyes.
options:
-h, --help show this help message and exit
--version print the version and exit
subcommands:
<command>
add add one note
list list notes
search find notes containing text
export write every note to standard output
remove delete notes by id
Examples:
notes add 'ring the dentist' --tag health --on 2026-03-01
echo 'from a pipe' | notes add - --tag inbox
notes list --format json | python3 -m json.tool
notes remove 3 --dry-run
Exit codes: 0 success, 1 refusal, 2 usage error.
exit: 0
$ python3 notes.py add --help
usage: notes add [-h] [--store PATH] [-v | -q] [-t TAG] [--on YYYY-MM-DD]
[--dry-run]
text
Add one note. Use '-' as the text to read it from standard input.
positional arguments:
text the note text, or '-' to read standard input
options:
-h, --help show this help message and exit
-t, --tag TAG attach a tag; repeat the option for more than one
--on YYYY-MM-DD the date to file the note under (default: today)
--dry-run say what would happen without writing anything
storage options:
where the notes live. Precedence: --store, then $NOTES_STORE, then
./notes.json
--store PATH path to the JSON store (default: $NOTES_STORE or
./notes.json)
output options:
-v, --verbose explain what is happening, on standard error
-q, --quiet print nothing on success
exit: 0
$ python3 notes.py list --help
usage: notes list [-h] [--store PATH] [-v | -q] [--format {table,json}]
[-t TAG] [--since YYYY-MM-DD] [-n N]
List notes, newest last.
options:
-h, --help show this help message and exit
--format {table,json}
output format (default: table)
-t, --tag TAG only notes with this tag
--since YYYY-MM-DD only notes on or after this date
-n, --limit N show at most N notes
storage options:
where the notes live. Precedence: --store, then $NOTES_STORE, then
./notes.json
--store PATH path to the JSON store (default: $NOTES_STORE or
./notes.json)
output options:
-v, --verbose explain what is happening, on standard error
-q, --quiet print nothing on success
exit: 0
$ python3 notes.py search --help
usage: notes search [-h] [--store PATH] [-v | -q] [-i] [--format {table,json}]
pattern
Find notes whose text contains PATTERN. Exits 1 when nothing matched.
positional arguments:
pattern the text to look for
options:
-h, --help show this help message and exit
-i, --ignore-case match regardless of case
--format {table,json}
output format (default: table)
storage options:
where the notes live. Precedence: --store, then $NOTES_STORE, then
./notes.json
--store PATH path to the JSON store (default: $NOTES_STORE or
./notes.json)
output options:
-v, --verbose explain what is happening, on standard error
-q, --quiet print nothing on success
exit: 0
$ python3 notes.py export --help
usage: notes export [-h] [--store PATH] [-v | -q] [--format {json,csv}]
Write every note to standard output so it can be piped or redirected.
options:
-h, --help show this help message and exit
--format {json,csv} output format (default: json)
storage options:
where the notes live. Precedence: --store, then $NOTES_STORE, then
./notes.json
--store PATH path to the JSON store (default: $NOTES_STORE or
./notes.json)
output options:
-v, --verbose explain what is happening, on standard error
-q, --quiet print nothing on success
exit: 0
$ python3 notes.py remove --help
usage: notes remove [-h] [--store PATH] [-v | -q] [--dry-run] ID [ID ...]
Delete one or more notes. Destructive, so it has --dry-run.
positional arguments:
ID note id to remove
options:
-h, --help show this help message and exit
--dry-run list what would be removed and leave the store untouched
storage options:
where the notes live. Precedence: --store, then $NOTES_STORE, then
./notes.json
--store PATH path to the JSON store (default: $NOTES_STORE or
./notes.json)
output options:
-v, --verbose explain what is happening, on standard error
-q, --quiet print nothing on success
exit: 0
$ python3 notes.py --version
notes 1.0.0
exit: 0
sample-run.txt
# A real session with the reference tool. Every line below was captured
# from an actual run; the store lives in a scratch directory.
$ python3 notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
added note 1
exit: 0
$ python3 notes.py add 'argparse turns a script into a tool' -t python -t writing --on 2026-03-02 --store notes.json
added note 2
exit: 0
$ echo 'from a pipe' | python3 notes.py add - --tag inbox --on 2026-03-03 --store notes.json
added note 3
exit: 0
$ python3 notes.py list --store notes.json
ID DATE TAGS TEXT
------------------------------------
1 2026-03-01 health ring the dentist
2 2026-03-02 python,writing argparse turns a script into a tool
3 2026-03-03 inbox from a pipe
exit: 0
$ python3 notes.py list --format json --store notes.json
[
{
"date": "2026-03-01",
"id": 1,
"tags": [
"health"
],
"text": "ring the dentist"
},
{
"date": "2026-03-02",
"id": 2,
"tags": [
"python",
"writing"
],
"text": "argparse turns a script into a tool"
},
{
"date": "2026-03-03",
"id": 3,
"tags": [
"inbox"
],
"text": "from a pipe"
}
]
exit: 0
$ python3 notes.py search PIPE -i --store notes.json
ID DATE TAGS TEXT
---------------------------
3 2026-03-03 inbox from a pipe
exit: 0
$ python3 notes.py search kangaroo --store notes.json
exit: 1
$ python3 notes.py export --format csv --store notes.json
id,date,tags,text
1,2026-03-01,health,ring the dentist
2,2026-03-02,python;writing,argparse turns a script into a tool
3,2026-03-03,inbox,from a pipe
exit: 0
# --- a destructive command, rehearsed first --------------------------
$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json
exit: 0
$ python3 notes.py remove 2 --dry-run --store notes.json
would remove note 2
dry run: 1 note(s) would be removed; notes.json was not touched
exit: 0
$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json
exit: 0
# The two hashes are identical: --dry-run really did write nothing.
$ python3 notes.py remove 2 --store notes.json
removed note 2
exit: 0
$ python3 notes.py list --store notes.json
ID DATE TAGS TEXT
----------------------------
1 2026-03-01 health ring the dentist
3 2026-03-03 inbox from a pipe
exit: 0
# --- the two streams are separate, which is what makes it composable -
$ python3 notes.py list --format json -v --store notes.json > result.json 2> chatter.txt
exit: 0
$ head -4 result.json # standard output: the RESULT, still valid JSON
[
{
"date": "2026-03-01",
"id": 1,
exit: 0
$ cat chatter.txt # standard error: the DIAGNOSTIC
2 note(s) from notes.json
exit: 0
# --- usage errors: exit 2, message on stderr, nothing on stdout ------
$ python3 notes.py add 'x' --on 2026-13-01 --store notes.json
usage: notes add [-h] [--store PATH] [-v | -q] [-t TAG] [--on YYYY-MM-DD]
[--dry-run]
text
notes add: error: argument --on: '2026-13-01' is not a date in YYYY-MM-DD form (for example 2026-03-01)
exit: 2
$ python3 notes.py add 'x' --on 2026-13-01 --store notes.json 2>/dev/null # stderr discarded
exit: 2 (and standard output was empty — nothing printed above)
$ python3 notes.py frobnicate --store notes.json
usage: notes [-h] [--version] <command> ...
notes: error: argument <command>: invalid choice: 'frobnicate' (choose from add, list, search, export, remove)
exit: 2
$ python3 notes.py list -v -q --store notes.json
usage: notes list [-h] [--store PATH] [-v | -q] [--format {table,json}]
[-t TAG] [--since YYYY-MM-DD] [-n N]
notes list: error: argument -q/--quiet: not allowed with argument -v/--verbose
exit: 2
$ python3 notes.py remove 99 --store notes.json
notes: no note with id 99 in notes.json
exit: 1
test-run.txt
# A real run of the Day 080 test suite on the authoring machine
# (macOS 26.5.1, Apple Silicon, Python 3.14.0, pytest 9.1.1, bash 3.2.57).
# The starter exercises are unfinished here, so the behavioural battery
# runs against the reference only. Finish all eight and it runs twice,
# giving 133 checks.
$ bash tests/run_tests.sh
Day 080 — A Tool You Would Actually Install
1. The reference tool (examples/notes.py)
ok: reference: --help exits 0
ok: reference: --help writes to standard output
ok: reference: --help writes nothing to standard error
ok: reference: --help names the 'add' subcommand
ok: reference: --help names the 'list' subcommand
ok: reference: --help names the 'search' subcommand
ok: reference: --help names the 'export' subcommand
ok: reference: --help names the 'remove' subcommand
ok: reference: usage line says 'notes', not the file name (prog=)
ok: reference: the epilog survives with its line breaks intact
ok: reference: 'add --help' shows the --on metavar
ok: reference: 'add --help' groups the storage options
ok: reference: --version prints 'notes 1.0.0' and exits 0
ok: reference: no subcommand at all exits 2
ok: reference: an unknown subcommand exits non-zero (2)
ok: reference: an unknown subcommand is explained on standard error
ok: reference: a mistyped option is refused, not ignored
ok: reference: -v and -q together exit 2, naming both options
ok: reference: a value outside choices= exits 2
ok: reference: the custom positive_int type rejects 0
ok: reference: a bad date exits non-zero (2)
ok: reference: a bad date exits exactly 2 — argparse's usage code
ok: reference: a bad date writes NOTHING to standard output
ok: reference: a bad date writes a message to standard error
ok: reference: the message quotes the value that was rejected
ok: reference: the message says what a date should look like
ok: reference: a refused command created no store file
ok: reference: the custom tag type rejects a tag with a space
ok: reference: a successful add exits 0, reports on stdout, is silent on stderr
ok: reference: --tag repeats (action='append')
ok: reference: 'add -' reads the note from a pipe
ok: reference: the piped text was really stored
ok: reference: '--' lets a note start with two dashes
ok: reference: 'list --format json' emits parseable JSON on stdout
ok: reference: 'list' defaults to the table format
ok: reference: '-n 1' limits the listing to one note
ok: reference: 'export --format csv' writes a header and four rows
ok: reference: -v chatter goes to stderr, leaving stdout valid JSON
ok: reference: -q succeeds in silence on both streams
ok: reference: search exits 0 when something matched
ok: reference: search exits 1 when nothing matched, printing nothing
ok: reference: '-i' matches regardless of case
ok: reference: 'remove --dry-run' exits 0
ok: reference: 'remove --dry-run' leaves the store BYTE-IDENTICAL
ok: reference: the dry run puts the ids on stdout, where they can be piped
ok: reference: the dry run puts its summary on stderr, where it is a diagnostic
ok: reference: a dry run still refuses a note that does not exist
ok: reference: control — a real remove DOES change the store
ok: reference: removing it twice is a refusal on stderr with exit 1
ok: reference: nargs='+' removes several notes in one command
ok: reference: $NOTES_STORE is used when --store is absent
ok: reference: --store beats $NOTES_STORE (flag wins)
ok: reference: with neither, the built-in ./notes.json default applies
ok: reference: 'export | python3' composes — stdout is machine-readable
ok: reference: closing the pipe early is not an error
ok: reference: a corrupt store is exit 1 on stderr, not a traceback
ok: reference: no Python traceback ever reaches the user
2. The hand-rolled parser, and why it is not enough
ok: examples/by_hand.py runs and exits 0
ok: the hand-rolled parser gets 4 of 8 ordinary command lines wrong
ok: it loses the value of --tag=shopping without saying so
ok: a missing option value gives an IndexError, not a usage message
ok: argparse handles --tag=shopping, which the hand-rolled parser dropped
ok: argparse turns a missing option value into a usage message and exit 2
3. In-process: parse_args with an explicit argv list
ok: the in-process pytest suite passes against examples/
ok: it reports passing tests ( 42 passed in 0.07s )
ok: a one-line break to prog= changes the help output, so the check bites
4. The starter
ok: starter/notes.py is valid Python and defines every required function
ok: starter/notes.py carries EXERCISE 1
ok: starter/notes.py carries EXERCISE 2
ok: starter/notes.py carries EXERCISE 3
ok: starter/notes.py carries EXERCISE 4
ok: starter/notes.py carries EXERCISE 5
ok: starter/notes.py carries EXERCISE 6
ok: starter/notes.py carries EXERCISE 7
ok: starter/notes.py carries EXERCISE 8
ok: starter/notes.py imports argparse
.. 6 exercise(s) still unfinished — the behavioural battery
will run against starter/notes.py once they are done.
76 checks, 0 failure(s).
exit: 0
Source files
examples/by_hand.py (3779 bytes)
#!/usr/bin/env python3
"""by_hand.py — what parsing sys.argv yourself actually costs.
Day 56 built a data-driven command-line tool with sys.argv, and for one
positional argument that is the right call: argparse would have been more
machinery than the job needed. This file shows the exact point where that
stops being true.
Run it with no arguments to watch a hand-rolled parser be put through eight
ordinary command lines, four of which it gets wrong. Nothing here is a straw
man: this is the shape people really write, right down to the manual index
arithmetic.
python3 by_hand.py
The point is not that hand-parsing is impossible. It is that everything the
hand-rolled version is missing — help text, a usage message, exit code 2,
short and long forms, `--flag=value`, clustered short flags, `--` to end
option parsing, type conversion, and the promise that an unknown option is
refused rather than ignored — is a line of argparse each.
"""
from __future__ import annotations
import sys
def parse_by_hand(argv: list[str]) -> dict[str, object]:
"""A hand-rolled parser for: notes add TEXT [--tag T] [--dry-run].
Written the way a hand-rolled parser is usually written: walk the list,
branch on what you find, and hope.
"""
text: str | None = None
tag: str | None = None
dry_run = False
i = 0
while i < len(argv):
item = argv[i]
if item == "--dry-run":
dry_run = True
elif item == "--tag":
i += 1
tag = argv[i] # IndexError if the user forgot the value
elif text is None:
text = item
i += 1
return {"text": text, "tag": tag, "dry_run": dry_run}
# Each case is (argv, correct?, comment). The `correct?` flag is stated
# explicitly rather than guessed from the text, because a count that depends
# on string-matching its own commentary is exactly the kind of quiet wrongness
# this file is about.
CASES: list[tuple[list[str], bool, str]] = [
(["buy milk"], True, "the happy path — this one works"),
(["buy milk", "--tag", "shopping"], True, "an option with a value — also fine"),
(["buy milk", "--tag", "shopping", "--dry-run"], True, "two options — still fine"),
(["buy milk", "--tag=shopping"], False, "--flag=value form: the tag is silently lost"),
(["buy milk", "-t", "shopping"], False, "short form: -t is swallowed as the text"),
(["buy milk", "--drynrun"], False, "a typo: accepted in silence, and does nothing"),
(["--dry-run", "buy milk"], True, "options before the text — right answer, by luck"),
(["buy milk", "--tag"], False, "a missing value: IndexError, and a traceback"),
]
def main() -> int:
print("A hand-rolled parser for: notes add TEXT [--tag T] [--dry-run]")
print()
wrong = 0
for argv, correct, comment in CASES:
printable = " ".join(repr(a) if " " in a else a for a in argv)
try:
result = parse_by_hand(argv)
got = f"text={result['text']!r} tag={result['tag']!r} dry_run={result['dry_run']}"
except IndexError:
got = "IndexError: list index out of range"
print(f" $ notes add {printable}")
print(f" {got}")
print(f" {comment}")
print()
if not correct:
wrong += 1
print(f"{len(CASES)} ordinary command lines, {wrong} of them handled wrongly.")
print()
print("None of these is exotic. Every one of them is something a user")
print("types on their first afternoon with your tool. And this parser")
print("still has no --help, no usage message, no exit code 2, no type")
print("conversion, and no way to refuse an option it does not know.")
return 0
if __name__ == "__main__":
sys.exit(main())
examples/notes.py (18631 bytes)
#!/usr/bin/env python3
"""notes — a small note-taking tool, built the way a real command should be.
This is the reference implementation for the Day 080 lab. Everything in it is
standard library: argparse, json, csv, os, sys, pathlib, datetime.
The shape to notice, because it is the shape of every serious Python CLI:
build_parser() builds the parser and NOTHING else — no I/O, no work
parse_args(argv) takes an EXPLICIT list, so tests never touch sys.argv
cmd_add / cmd_list / one handler per subcommand, bound by set_defaults
cmd_search / ... so dispatch is `args.func(args, streams)`
main(argv, streams) wires them together and returns an exit code
Streams are injected rather than reached for, so the same code can be driven
by a test with io.StringIO in place of the terminal. That is Day 74's boundary
lesson applied to standard input and standard output.
Exit codes:
0 the command did what it said
1 a runtime refusal (no such note, unreadable store)
2 a usage error (argparse's own convention, and we keep it)
"""
from __future__ import annotations
import argparse
import csv
import json
import os
import sys
from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path
from typing import Any, Callable, Sequence, TextIO
__version__ = "1.0.0"
DEFAULT_STORE = "notes.json"
STORE_ENV_VAR = "NOTES_STORE"
# ---------------------------------------------------------------------------
# Errors
# ---------------------------------------------------------------------------
class NotesError(Exception):
"""A refusal stated in the language of the tool, not of the machinery."""
# ---------------------------------------------------------------------------
# Streams — the three surfaces every command-line program stands on
# ---------------------------------------------------------------------------
@dataclass
class Streams:
"""Standard input, output and error, passed in rather than imported.
A handler that writes to `streams.stdout` can be tested by handing it an
io.StringIO. A handler that calls `print()` can only be tested by capturing
the process. The difference is one parameter.
"""
stdin: TextIO
stdout: TextIO
stderr: TextIO
@classmethod
def real(cls) -> "Streams":
return cls(stdin=sys.stdin, stdout=sys.stdout, stderr=sys.stderr)
def stdin_is_a_terminal(self) -> bool:
"""True when nothing is piped in — so the tool would hang waiting."""
try:
return self.stdin.isatty()
except (AttributeError, ValueError):
return False
# ---------------------------------------------------------------------------
# Custom argument types — conversion and validation in one place
# ---------------------------------------------------------------------------
def iso_date(text: str) -> date:
"""Convert YYYY-MM-DD to a date, or explain precisely why it cannot.
Raising ArgumentTypeError is what makes argparse print a usage message on
standard error and exit 2. Raising ValueError would give the user a
traceback, which is a bug report addressed to the wrong person.
"""
try:
return datetime.strptime(text, "%Y-%m-%d").date()
except ValueError:
raise argparse.ArgumentTypeError(
f"{text!r} is not a date in YYYY-MM-DD form (for example 2026-03-01)"
) from None
def positive_int(text: str) -> int:
"""Convert to an int that is at least 1."""
try:
value = int(text)
except ValueError:
raise argparse.ArgumentTypeError(f"{text!r} is not a whole number") from None
if value < 1:
raise argparse.ArgumentTypeError(f"{value} is not at least 1")
return value
def tag_name(text: str) -> str:
"""A tag is lower case, non-empty, and has no spaces or commas."""
cleaned = text.strip().lower()
if not cleaned:
raise argparse.ArgumentTypeError("a tag cannot be empty")
if any(ch in cleaned for ch in " ,\t"):
raise argparse.ArgumentTypeError(
f"{text!r} is not a tag: tags contain no spaces or commas"
)
return cleaned
# ---------------------------------------------------------------------------
# The store — a small JSON file, kept behind two functions
# ---------------------------------------------------------------------------
def load_store(path: Path) -> dict[str, Any]:
"""Read the store, or return an empty one if the file is not there yet."""
if not path.exists():
return {"version": 1, "notes": []}
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise NotesError(f"{path} is not valid JSON ({exc.msg}, line {exc.lineno})") from None
except OSError as exc:
raise NotesError(f"cannot read {path}: {exc.strerror}") from None
if not isinstance(data, dict) or not isinstance(data.get("notes"), list):
raise NotesError(f"{path} does not look like a notes store")
return data
def save_store(path: Path, data: dict[str, Any]) -> None:
"""Write the store back, formatted so a human can read the diff."""
text = json.dumps(data, indent=2, ensure_ascii=False, sort_keys=True) + "\n"
try:
path.write_text(text, encoding="utf-8")
except OSError as exc:
raise NotesError(f"cannot write {path}: {exc.strerror}") from None
def next_id(notes: list[dict[str, Any]]) -> int:
return max((int(note["id"]) for note in notes), default=0) + 1
def store_path(args: argparse.Namespace) -> Path:
"""Resolve the store path using the precedence users expect.
flag beats environment beats built-in default. The flag's own default is
None precisely so that "the user did not say" is distinguishable from
"the user said the default".
"""
if args.store is not None:
return Path(args.store)
from_env = os.environ.get(STORE_ENV_VAR)
if from_env:
return Path(from_env)
return Path(DEFAULT_STORE)
# ---------------------------------------------------------------------------
# Rendering
# ---------------------------------------------------------------------------
def render_table(notes: list[dict[str, Any]], stream: TextIO) -> None:
if not notes:
return
id_width = max(2, max(len(str(note["id"])) for note in notes))
tag_width = max(4, max(len(",".join(note["tags"])) for note in notes))
header = f"{'ID'.rjust(id_width)} {'DATE':10} {'TAGS'.ljust(tag_width)} TEXT"
stream.write(header + "\n")
stream.write("-" * len(header) + "\n")
for note in notes:
tags = ",".join(note["tags"])
stream.write(
f"{str(note['id']).rjust(id_width)} {note['date']:10} "
f"{tags.ljust(tag_width)} {note['text']}\n"
)
def render_json(notes: list[dict[str, Any]], stream: TextIO) -> None:
json.dump(notes, stream, indent=2, ensure_ascii=False, sort_keys=True)
stream.write("\n")
def render_csv(notes: list[dict[str, Any]], stream: TextIO) -> None:
writer = csv.writer(stream, lineterminator="\n")
writer.writerow(["id", "date", "tags", "text"])
for note in notes:
writer.writerow([note["id"], note["date"], ";".join(note["tags"]), note["text"]])
# ---------------------------------------------------------------------------
# Subcommand handlers — each returns an exit code, each takes its streams
# ---------------------------------------------------------------------------
def cmd_add(args: argparse.Namespace, streams: Streams) -> int:
if args.text == "-":
if streams.stdin_is_a_terminal():
args.parser.error(
"reading from standard input was requested with '-', "
"but standard input is a terminal; pipe something in"
)
text = streams.stdin.read().strip()
if not text:
raise NotesError("standard input was empty, so there is nothing to add")
else:
text = args.text.strip()
if not text:
raise NotesError("a note cannot be empty")
when = args.on if args.on is not None else date.today()
path = store_path(args)
data = load_store(path)
note = {
"id": next_id(data["notes"]),
"date": when.isoformat(),
"tags": sorted(set(args.tag or [])),
"text": text,
}
data["notes"].append(note)
if args.dry_run:
if not args.quiet:
streams.stdout.flush()
streams.stderr.write(
f"dry run: would add note {note['id']} to {path} (nothing was written)\n"
)
return 0
save_store(path, data)
if not args.quiet:
streams.stdout.write(f"added note {note['id']}\n")
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"store: {path}\n")
return 0
def select_notes(args: argparse.Namespace, notes: list[dict[str, Any]]) -> list[dict[str, Any]]:
chosen = notes
if args.tag:
wanted = set(args.tag)
chosen = [n for n in chosen if wanted.issubset(set(n["tags"]))]
if getattr(args, "since", None) is not None:
chosen = [n for n in chosen if n["date"] >= args.since.isoformat()]
if getattr(args, "limit", None) is not None:
chosen = chosen[-args.limit :]
return chosen
def cmd_list(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
notes = select_notes(args, load_store(path)["notes"])
if args.format == "json":
render_json(notes, streams.stdout)
else:
render_table(notes, streams.stdout)
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"{len(notes)} note(s) from {path}\n")
return 0
def cmd_search(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
notes = load_store(path)["notes"]
needle = args.pattern.lower() if args.ignore_case else args.pattern
matches = [
note
for note in notes
if needle in (note["text"].lower() if args.ignore_case else note["text"])
]
if args.format == "json":
render_json(matches, streams.stdout)
else:
render_table(matches, streams.stdout)
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"{len(matches)} of {len(notes)} note(s) matched\n")
return 0 if matches else 1
def cmd_export(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
notes = load_store(path)["notes"]
if args.format == "csv":
render_csv(notes, streams.stdout)
else:
render_json(notes, streams.stdout)
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"exported {len(notes)} note(s) as {args.format}\n")
return 0
def cmd_remove(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
data = load_store(path)
present = {int(note["id"]) for note in data["notes"]}
missing = [str(i) for i in args.ids if i not in present]
if missing:
raise NotesError(f"no note with id {', '.join(missing)} in {path}")
doomed = sorted(set(args.ids))
if args.dry_run:
for note_id in doomed:
streams.stdout.write(f"would remove note {note_id}\n")
streams.stdout.flush()
streams.stderr.write(
f"dry run: {len(doomed)} note(s) would be removed; {path} was not touched\n"
)
return 0
data["notes"] = [n for n in data["notes"] if int(n["id"]) not in set(doomed)]
save_store(path, data)
if not args.quiet:
for note_id in doomed:
streams.stdout.write(f"removed note {note_id}\n")
return 0
# ---------------------------------------------------------------------------
# The parser — a declaration of the tool's whole interface, in one place
# ---------------------------------------------------------------------------
def common_options() -> argparse.ArgumentParser:
"""Options shared by every subcommand.
`add_help=False` matters: this parser is only ever used as a parent, and
without it every subcommand would try to define -h twice and crash.
"""
parent = argparse.ArgumentParser(add_help=False)
storage = parent.add_argument_group(
"storage options",
f"where the notes live. Precedence: --store, then ${STORE_ENV_VAR}, "
f"then ./{DEFAULT_STORE}",
)
storage.add_argument(
"--store",
metavar="PATH",
default=None,
help=f"path to the JSON store (default: ${STORE_ENV_VAR} or ./{DEFAULT_STORE})",
)
noise = parent.add_argument_group("output options")
loudness = noise.add_mutually_exclusive_group()
loudness.add_argument(
"-v",
"--verbose",
action="store_true",
help="explain what is happening, on standard error",
)
loudness.add_argument(
"-q",
"--quiet",
action="store_true",
help="print nothing on success",
)
return parent
def build_parser() -> argparse.ArgumentParser:
"""Build the parser. This function does no work and touches no file."""
parent = common_options()
parser = argparse.ArgumentParser(
prog="notes",
description="Keep short notes in a JSON file you can read with your own eyes.",
epilog=(
"Examples:\n"
" notes add 'ring the dentist' --tag health --on 2026-03-01\n"
" echo 'from a pipe' | notes add - --tag inbox\n"
" notes list --format json | python3 -m json.tool\n"
" notes remove 3 --dry-run\n"
"\n"
"Exit codes: 0 success, 1 refusal, 2 usage error."
),
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--version",
action="version",
version=f"%(prog)s {__version__}",
help="print the version and exit",
)
subcommands = parser.add_subparsers(
title="subcommands",
dest="command",
metavar="<command>",
required=True,
)
# --- add ---------------------------------------------------------------
add = subcommands.add_parser(
"add",
parents=[parent],
help="add one note",
description="Add one note. Use '-' as the text to read it from standard input.",
)
add.add_argument("text", help="the note text, or '-' to read standard input")
add.add_argument(
"-t",
"--tag",
action="append",
type=tag_name,
metavar="TAG",
help="attach a tag; repeat the option for more than one",
)
add.add_argument(
"--on",
type=iso_date,
metavar="YYYY-MM-DD",
help="the date to file the note under (default: today)",
)
add.add_argument(
"--dry-run",
action="store_true",
help="say what would happen without writing anything",
)
add.set_defaults(func=cmd_add, parser=add)
# --- list --------------------------------------------------------------
listing = subcommands.add_parser(
"list",
parents=[parent],
help="list notes",
description="List notes, newest last.",
)
listing.add_argument(
"--format",
choices=["table", "json"],
default="table",
help="output format (default: table)",
)
listing.add_argument(
"-t", "--tag", action="append", type=tag_name, metavar="TAG", help="only notes with this tag"
)
listing.add_argument("--since", type=iso_date, metavar="YYYY-MM-DD", help="only notes on or after this date")
listing.add_argument("-n", "--limit", type=positive_int, metavar="N", help="show at most N notes")
listing.set_defaults(func=cmd_list, parser=listing)
# --- search ------------------------------------------------------------
search = subcommands.add_parser(
"search",
parents=[parent],
help="find notes containing text",
description="Find notes whose text contains PATTERN. Exits 1 when nothing matched.",
)
search.add_argument("pattern", help="the text to look for")
search.add_argument("-i", "--ignore-case", action="store_true", help="match regardless of case")
search.add_argument(
"--format", choices=["table", "json"], default="table", help="output format (default: table)"
)
search.set_defaults(func=cmd_search, parser=search)
# --- export ------------------------------------------------------------
export = subcommands.add_parser(
"export",
parents=[parent],
help="write every note to standard output",
description="Write every note to standard output so it can be piped or redirected.",
)
export.add_argument(
"--format", choices=["json", "csv"], default="json", help="output format (default: json)"
)
export.set_defaults(func=cmd_export, parser=export)
# --- remove ------------------------------------------------------------
remove = subcommands.add_parser(
"remove",
parents=[parent],
help="delete notes by id",
description="Delete one or more notes. Destructive, so it has --dry-run.",
)
remove.add_argument("ids", nargs="+", type=positive_int, metavar="ID", help="note id to remove")
remove.add_argument(
"--dry-run",
action="store_true",
help="list what would be removed and leave the store untouched",
)
remove.set_defaults(func=cmd_remove, parser=remove)
return parser
def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
"""Parse an EXPLICIT argument list.
Defaulting to None (which argparse turns into sys.argv[1:]) keeps the real
program convenient, while every test passes its own list and never has to
monkey-patch a global.
"""
return build_parser().parse_args(argv)
def main(argv: Sequence[str] | None = None, streams: Streams | None = None) -> int:
streams = streams or Streams.real()
args = parse_args(argv)
handler: Callable[[argparse.Namespace, Streams], int] = args.func
try:
return handler(args, streams)
except NotesError as exc:
streams.stderr.write(f"notes: {exc}\n")
return 1
except BrokenPipeError:
# `notes export | head -2` closes the pipe early. That is not an error.
return 0
except KeyboardInterrupt:
streams.stderr.write("notes: interrupted\n")
return 130
if __name__ == "__main__":
sys.exit(main())
metadata.yml (1259 bytes)
lesson_id: D080
day: 80
kind: python-program
languages: [python]
setup_commands:
- cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
- python3 --version
- python3 -m venv .venv
- .venv/bin/pip install -r requirements/requirements.txt
run_commands:
- python3 examples/by_hand.py
- python3 examples/notes.py --help
- python3 examples/notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
- echo 'from a pipe' | python3 examples/notes.py add - --tag inbox --on 2026-03-03 --store notes.json
- python3 examples/notes.py list --format json --store notes.json
- python3 examples/notes.py remove 2 --dry-run --store notes.json
- python3 starter/notes.py --help
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -f notes.json
- 'git checkout -- starter/ # optional: reset your work'
requires_network: true
requires_api_key: false
estimated_minutes: 30
last_executed: '2026-07-19'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1, bash 3.2.57 — bash tests/run_tests.sh -> 76 checks, 0 failure(s), exit 0 with the starter exercises unfinished (133 checks, 0 failure(s), exit 0 once all eight are completed); the in-process suite reports 42 passed'
requirements/README.md (2848 bytes)
# Dependencies — Day 080 lab
**One package, and it is only for the tests.**
The tool you build in this lab has **no third-party dependencies at all**.
`argparse` has been in the Python standard library since Python 3.2, released
in 2011, and so has everything else the tool touches. That is not an accident
of this lab's design — it is one of the strongest practical arguments for
argparse, and it is worth stating plainly rather than burying: a command-line
tool written with argparse can be handed to anyone with a Python interpreter
and it runs. No install step, no version pin, no network.
| Module | Used in | Why |
| --- | --- | --- |
| `argparse` | `notes.py` | the whole command-line interface: parser, subcommands, types, choices, groups, help |
| `json` | `notes.py` | the store format, and the `--format json` output |
| `csv` | `notes.py` | the `--format csv` export, quoted correctly by `csv.writer` |
| `os` | `notes.py` | reading `$NOTES_STORE` for the configuration-precedence rule |
| `sys` | `notes.py` | the three real streams and the process exit code |
| `pathlib` | `notes.py` | reading and writing the store file |
| `datetime` | `notes.py` | the `--on` date, converted by the custom `iso_date` type |
| `dataclasses` | `notes.py` | the small `Streams` record that carries stdin, stdout and stderr |
## What is in requirements.txt, and why
```text
pytest==9.1.1
```
`pytest` is needed only by `tests/test_parser.py`, the in-process half of the
suite — the part that calls `parse_args(["add", "hello"])` directly and
asserts on the resulting namespace without starting a process. You met pytest
on Day 71 and used it for the whole of Week 11.
pytest is free and open source (MIT licence). The exact version above is the
one the captured output in `expected-output/` came from; any recent pytest
will do, and the runner accepts an existing installation.
## Install
```bash
cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
```
`tests/run_tests.sh` looks for pytest in three places, in order: the `PYTEST`
environment variable, this lab's `.venv/bin/`, and then whatever is on your
`PATH`. If it finds none it prints the two commands above and exits non-zero —
it never skips silently, because a suite that quietly does nothing is worse
than a suite that fails.
If you already have pytest somewhere else:
```bash
PYTEST=/path/to/pytest bash tests/run_tests.sh
```
## No network, no account, no key
Nothing in this lab reaches the network at any point, including the install
step if you already have pytest. There is no account to create, no API key,
and nothing to pay for. The alternatives discussed in the lesson — `click`
and `typer` — are also free and open source, and neither is needed here.
requirements/requirements.txt (305 bytes)
# Day 080 lab — dependencies
#
# The TOOL you build needs nothing: argparse, json, csv, os, sys, pathlib and
# datetime are all Python standard library. The line below is for the TESTS.
#
# Install with:
# python3 -m venv .venv
# .venv/bin/pip install -r requirements/requirements.txt
pytest==9.1.1
starter/notes.py (21268 bytes)
#!/usr/bin/env python3
"""notes — YOUR working file for the Day 080 lab.
Eight numbered exercises build this into a tool you would actually install.
Everything that is NOT about the command-line interface — the JSON store, the
table and CSV renderers, the error type — is already written for you, because
today is about the interface, not about the storage.
Work top to bottom. After each exercise run:
bash tests/run_tests.sh
The suite runs a structural pass while exercises remain, and switches to the
full behavioural battery — the same one the reference passes — the moment the
last NotImplementedError is gone.
The exercises, at a glance:
1. iso_date a custom type= that converts and validates a date
2. positive_int a second custom type=, for --limit and note ids
3. common_options an argument group and a mutually exclusive group
4. build_parser the root parser: prog, description, epilog, --version
5. build_parser the `add` subparser and its set_defaults dispatch
6. build_parser the list / search / export / remove subparsers
7. cmd_add reading the note from standard input when text is '-'
8. cmd_remove --dry-run: say what would happen, write nothing
Compare with examples/notes.py only AFTER you have tried. Reading the answer
first turns a two-hour lesson into a five-minute copy.
"""
from __future__ import annotations
import argparse
import csv
import json
import os
import sys
from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path
from typing import Any, Callable, Sequence, TextIO
__version__ = "1.0.0"
DEFAULT_STORE = "notes.json"
STORE_ENV_VAR = "NOTES_STORE"
# ===========================================================================
# PROVIDED — you do not need to change anything between here and EXERCISE 1
# ===========================================================================
class NotesError(Exception):
"""A refusal stated in the language of the tool, not of the machinery."""
@dataclass
class Streams:
"""Standard input, output and error, passed in rather than imported."""
stdin: TextIO
stdout: TextIO
stderr: TextIO
@classmethod
def real(cls) -> "Streams":
return cls(stdin=sys.stdin, stdout=sys.stdout, stderr=sys.stderr)
def stdin_is_a_terminal(self) -> bool:
"""True when nothing is piped in — so the tool would hang waiting."""
try:
return self.stdin.isatty()
except (AttributeError, ValueError):
return False
def load_store(path: Path) -> dict[str, Any]:
if not path.exists():
return {"version": 1, "notes": []}
try:
data = json.loads(path.read_text(encoding="utf-8"))
except json.JSONDecodeError as exc:
raise NotesError(f"{path} is not valid JSON ({exc.msg}, line {exc.lineno})") from None
except OSError as exc:
raise NotesError(f"cannot read {path}: {exc.strerror}") from None
if not isinstance(data, dict) or not isinstance(data.get("notes"), list):
raise NotesError(f"{path} does not look like a notes store")
return data
def save_store(path: Path, data: dict[str, Any]) -> None:
text = json.dumps(data, indent=2, ensure_ascii=False, sort_keys=True) + "\n"
try:
path.write_text(text, encoding="utf-8")
except OSError as exc:
raise NotesError(f"cannot write {path}: {exc.strerror}") from None
def next_id(notes: list[dict[str, Any]]) -> int:
return max((int(note["id"]) for note in notes), default=0) + 1
def store_path(args: argparse.Namespace) -> Path:
"""Precedence: --store flag, then $NOTES_STORE, then ./notes.json."""
if args.store is not None:
return Path(args.store)
from_env = os.environ.get(STORE_ENV_VAR)
if from_env:
return Path(from_env)
return Path(DEFAULT_STORE)
def render_table(notes: list[dict[str, Any]], stream: TextIO) -> None:
if not notes:
return
id_width = max(2, max(len(str(note["id"])) for note in notes))
tag_width = max(4, max(len(",".join(note["tags"])) for note in notes))
header = f"{'ID'.rjust(id_width)} {'DATE':10} {'TAGS'.ljust(tag_width)} TEXT"
stream.write(header + "\n")
stream.write("-" * len(header) + "\n")
for note in notes:
tags = ",".join(note["tags"])
stream.write(
f"{str(note['id']).rjust(id_width)} {note['date']:10} "
f"{tags.ljust(tag_width)} {note['text']}\n"
)
def render_json(notes: list[dict[str, Any]], stream: TextIO) -> None:
json.dump(notes, stream, indent=2, ensure_ascii=False, sort_keys=True)
stream.write("\n")
def render_csv(notes: list[dict[str, Any]], stream: TextIO) -> None:
writer = csv.writer(stream, lineterminator="\n")
writer.writerow(["id", "date", "tags", "text"])
for note in notes:
writer.writerow([note["id"], note["date"], ";".join(note["tags"]), note["text"]])
def tag_name(text: str) -> str:
"""A tag is lower case, non-empty, and has no spaces or commas."""
cleaned = text.strip().lower()
if not cleaned:
raise argparse.ArgumentTypeError("a tag cannot be empty")
if any(ch in cleaned for ch in " ,\t"):
raise argparse.ArgumentTypeError(
f"{text!r} is not a tag: tags contain no spaces or commas"
)
return cleaned
# ===========================================================================
# EXERCISE 1 — a custom type= that converts AND validates
# ===========================================================================
#
# argparse's `type=` is not only a converter; it is your validation hook. A
# function passed as `type=` receives the raw string, returns the converted
# value, and raises argparse.ArgumentTypeError when the string is unusable.
# argparse turns that exception into a usage message on STANDARD ERROR and
# exits 2. Raise a plain ValueError instead and the user gets a traceback.
#
# Write iso_date so that:
# iso_date("2026-03-01") -> datetime.date(2026, 3, 1)
# iso_date("2026-13-01") -> raises argparse.ArgumentTypeError
# iso_date("tomorrow") -> raises argparse.ArgumentTypeError
#
# Hint: datetime.strptime(text, "%Y-%m-%d").date() raises ValueError on bad
# input. Catch it and re-raise as ArgumentTypeError with a message naming both
# the offending value and the form you wanted. `from None` suppresses the
# chained traceback, which the user does not need to see.
#
# Check it:
# python3 -c "import notes; print(notes.iso_date('2026-03-01'))"
def iso_date(text: str) -> date:
raise NotImplementedError("EXERCISE 1: convert YYYY-MM-DD or raise ArgumentTypeError")
# ===========================================================================
# EXERCISE 2 — a second custom type, for counts and ids
# ===========================================================================
#
# positive_int("3") -> 3
# positive_int("0") -> raises argparse.ArgumentTypeError
# positive_int("-2") -> raises argparse.ArgumentTypeError
# positive_int("abc") -> raises argparse.ArgumentTypeError
#
# Two different failures, two different messages: "is not a whole number" and
# "is not at least 1". A user who mistypes should not have to guess which.
def positive_int(text: str) -> int:
raise NotImplementedError("EXERCISE 2: parse a whole number of at least 1")
# ===========================================================================
# PROVIDED — the handlers, except the two bodies you write in 7 and 8
# ===========================================================================
def cmd_add(args: argparse.Namespace, streams: Streams) -> int:
if args.text == "-":
# ===================================================================
# EXERCISE 7 — read the note from standard input
# ===================================================================
#
# `-` meaning "standard input" is a convention older than Python, and
# following it is what lets your tool live inside a pipeline:
#
# echo 'from a pipe' | python3 notes.py add - --tag inbox
#
# Two things must happen here.
#
# (a) TERMINAL DETECTION. If standard input is a terminal, nothing is
# piped in and reading would hang with no explanation. Call
# streams.stdin_is_a_terminal() and, when it is True, call
# args.parser.error("...")
# which prints usage on standard error and exits 2. Say what the
# user should have done, not just that they were wrong.
#
# (b) READ AND CHECK. Otherwise read streams.stdin.read(), strip it,
# and raise NotesError if it is empty — an empty pipe is a
# refusal, not a note.
#
# Assign the result to `text`.
raise NotImplementedError("EXERCISE 7: read the note from standard input")
else:
text = args.text.strip()
if not text:
raise NotesError("a note cannot be empty")
when = args.on if args.on is not None else date.today()
path = store_path(args)
data = load_store(path)
note = {
"id": next_id(data["notes"]),
"date": when.isoformat(),
"tags": sorted(set(args.tag or [])),
"text": text,
}
data["notes"].append(note)
if args.dry_run:
if not args.quiet:
streams.stdout.flush()
streams.stderr.write(
f"dry run: would add note {note['id']} to {path} (nothing was written)\n"
)
return 0
save_store(path, data)
if not args.quiet:
streams.stdout.write(f"added note {note['id']}\n")
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"store: {path}\n")
return 0
def select_notes(args: argparse.Namespace, notes: list[dict[str, Any]]) -> list[dict[str, Any]]:
chosen = notes
if args.tag:
wanted = set(args.tag)
chosen = [n for n in chosen if wanted.issubset(set(n["tags"]))]
if getattr(args, "since", None) is not None:
chosen = [n for n in chosen if n["date"] >= args.since.isoformat()]
if getattr(args, "limit", None) is not None:
chosen = chosen[-args.limit :]
return chosen
def cmd_list(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
notes = select_notes(args, load_store(path)["notes"])
if args.format == "json":
render_json(notes, streams.stdout)
else:
render_table(notes, streams.stdout)
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"{len(notes)} note(s) from {path}\n")
return 0
def cmd_search(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
notes = load_store(path)["notes"]
needle = args.pattern.lower() if args.ignore_case else args.pattern
matches = [
note
for note in notes
if needle in (note["text"].lower() if args.ignore_case else note["text"])
]
if args.format == "json":
render_json(matches, streams.stdout)
else:
render_table(matches, streams.stdout)
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"{len(matches)} of {len(notes)} note(s) matched\n")
# grep's convention: nothing found is exit 1, not an error message.
return 0 if matches else 1
def cmd_export(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
notes = load_store(path)["notes"]
if args.format == "csv":
render_csv(notes, streams.stdout)
else:
render_json(notes, streams.stdout)
if args.verbose:
streams.stdout.flush()
streams.stderr.write(f"exported {len(notes)} note(s) as {args.format}\n")
return 0
def cmd_remove(args: argparse.Namespace, streams: Streams) -> int:
path = store_path(args)
data = load_store(path)
present = {int(note["id"]) for note in data["notes"]}
missing = [str(i) for i in args.ids if i not in present]
if missing:
raise NotesError(f"no note with id {', '.join(missing)} in {path}")
doomed = sorted(set(args.ids))
if args.dry_run:
# =====================================================================
# EXERCISE 8 — --dry-run on the one destructive command
# =====================================================================
#
# A dry run is a promise: the tool tells you exactly what it would do
# and changes nothing. Two rules make that promise keepable.
#
# 1. Do the READS and the CHECKS for real. Notice that the
# "no note with id ..." check above already ran — a dry run that
# skipped validation would happily promise an impossible deletion.
# 2. Never reach the write. Return before save_store is called.
#
# Write, for each id in `doomed`, one line on STANDARD OUTPUT:
# would remove note 2
# then one summary line on STANDARD ERROR:
# dry run: 1 note(s) would be removed; <path> was not touched
# then return 0.
#
# The split is deliberate: the ids are the RESULT (a person could pipe
# them into another command), the summary is a DIAGNOSTIC. Call
# streams.stdout.flush() before writing to stderr so the two arrive in
# the order you wrote them even when standard output is a pipe.
#
# The test suite checks the store is byte-for-byte identical
# afterwards. That is the only check that can prove a dry run is real.
raise NotImplementedError("EXERCISE 8: report what would happen, write nothing")
data["notes"] = [n for n in data["notes"] if int(n["id"]) not in set(doomed)]
save_store(path, data)
if not args.quiet:
for note_id in doomed:
streams.stdout.write(f"removed note {note_id}\n")
return 0
# ===========================================================================
# EXERCISE 3 — shared options: an argument group and a mutually exclusive pair
# ===========================================================================
#
# Every subcommand needs --store, --verbose and --quiet. Rather than repeating
# them five times, define them once on a PARENT parser and pass parents=[...]
# to each subparser. Note add_help=False: without it, every subcommand would
# try to define -h twice and argparse would raise at import time.
#
# Build, inside this function:
#
# (a) an ARGUMENT GROUP called "storage options", created with
# parent.add_argument_group(title, description). Groups do not change
# parsing at all — they only change how --help is laid out, which is
# reason enough, because your help output is the documentation most
# users will ever read. Give it --store with metavar="PATH",
# default=None, and a help string naming the precedence.
#
# default=None is not laziness. It is how you tell "the user did not
# say" apart from "the user said the default" — which is exactly what
# store_path() above needs in order to consult $NOTES_STORE.
#
# (b) a MUTUALLY EXCLUSIVE GROUP holding -v/--verbose and -q/--quiet, both
# action="store_true". Created with
# some_group.add_mutually_exclusive_group(). Passing both then fails
# with exit code 2 and a message naming both options — argparse does
# that for you, and a hand-rolled `if verbose and quiet` would not
# produce a usage message.
#
# Return the parent parser.
def common_options() -> argparse.ArgumentParser:
parent = argparse.ArgumentParser(add_help=False)
raise NotImplementedError("EXERCISE 3: add the storage group and the verbose/quiet pair")
# ===========================================================================
# EXERCISES 4, 5 and 6 — the parser itself
# ===========================================================================
def build_parser() -> argparse.ArgumentParser:
parent = common_options()
# =======================================================================
# EXERCISE 4 — the root parser
# =======================================================================
#
# Create argparse.ArgumentParser with, at minimum:
# prog="notes" so usage lines say `notes`, not `notes.py`
# description=... one sentence, shown at the top of --help
# epilog=... worked examples, shown at the bottom
# formatter_class=argparse.RawDescriptionHelpFormatter
# so your epilog's line breaks survive
#
# Then add --version with action="version" and
# version=f"%(prog)s {__version__}". `%(prog)s` is argparse's own
# substitution, not an f-string field — it expands to the prog name.
#
# Users expect -h, --help and --version to exist. Two of the three you get
# for free; the third is one line. Not having them is the first thing that
# makes a tool feel homemade.
raise NotImplementedError("EXERCISE 4: build the root parser and add --version")
# =======================================================================
# EXERCISE 5 — subcommands and the dispatch pattern
# =======================================================================
#
# subcommands = parser.add_subparsers(
# title="subcommands", dest="command", metavar="<command>", required=True,
# )
#
# required=True is what makes bare `notes` exit 2 with a usage message
# instead of crashing with AttributeError when nothing set args.func.
#
# Then, for `add`:
#
# add = subcommands.add_parser("add", parents=[parent],
# help="add one note",
# description="...")
# add.add_argument("text", help="the note text, or '-' to read stdin")
# add.add_argument("-t", "--tag", action="append", type=tag_name,
# metavar="TAG", help="...")
# add.add_argument("--on", type=iso_date, metavar="YYYY-MM-DD", help="...")
# add.add_argument("--dry-run", action="store_true", help="...")
# add.set_defaults(func=cmd_add, parser=add)
#
# That last line is the whole dispatch pattern, and it is the single most
# useful technique in this lesson. set_defaults stores arbitrary values on
# the resulting namespace, so after parsing, `args.func` IS the right
# handler. main() calls it without a single if-statement about which
# subcommand ran. Storing `parser=add` as well gives the handler access to
# its own parser so it can call parser.error() — which is how EXERCISE 7
# reports a usage problem it could only detect at run time.
#
# Note also: `--dry-run` on the command line becomes `args.dry_run`.
# argparse replaces dashes with underscores, because a dash is not legal
# in a Python identifier.
# =======================================================================
# EXERCISE 6 — the remaining four subcommands
# =======================================================================
#
# list --format {table,json} (default "table"), -t/--tag (append,
# type=tag_name), --since (type=iso_date),
# -n/--limit (type=positive_int) -> cmd_list
# search positional `pattern`; -i/--ignore-case (store_true);
# --format {table,json} -> cmd_search
# export --format {json,csv} (default "json") -> cmd_export
# remove positional `ids` with nargs="+" and type=positive_int;
# --dry-run (store_true) -> cmd_remove
#
# Every one takes parents=[parent] and ends with set_defaults(func=...,
# parser=...).
#
# `choices=` is worth pausing on: it validates, it prints the legal values
# in the usage line, and a wrong value exits 2 with a message listing what
# was allowed. Three jobs, one keyword.
#
# nargs="+" means one or more, collected into a list — so
# `notes remove 2 5 9` deletes three notes with no extra parsing.
#
# Finally: return parser
def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
"""Parse an EXPLICIT argument list.
Defaulting to None (which argparse turns into sys.argv[1:]) keeps the real
program convenient, while every test passes its own list and never has to
monkey-patch a global. This is Day 74's injection argument, applied to the
command line.
"""
return build_parser().parse_args(argv)
def main(argv: Sequence[str] | None = None, streams: Streams | None = None) -> int:
streams = streams or Streams.real()
args = parse_args(argv)
handler: Callable[[argparse.Namespace, Streams], int] = args.func
try:
return handler(args, streams)
except NotesError as exc:
streams.stderr.write(f"notes: {exc}\n")
return 1
except BrokenPipeError:
return 0
except KeyboardInterrupt:
streams.stderr.write("notes: interrupted\n")
return 130
if __name__ == "__main__":
sys.exit(main())
tests/conftest.py (1128 bytes)
"""Make the tool importable in-process.
Which copy is tested is chosen by the NOTES_DIR environment variable, so the
same suite can be pointed at examples/ (the reference) or at starter/ (your
work) without editing a line. `bash tests/run_tests.sh` sets it for you.
Importing the module rather than launching a process is the whole point of
the in-process tests: they exercise `parse_args(argv)` with an explicit list
and `main(argv, streams)` with io.StringIO streams, which is only possible
because neither function reaches for sys.argv or sys.stdout on its own.
"""
from __future__ import annotations
import os
import sys
from pathlib import Path
import pytest
LAB_DIR = Path(__file__).resolve().parent.parent
NOTES_DIR = LAB_DIR / os.environ.get("NOTES_DIR", "examples")
sys.path.insert(0, str(NOTES_DIR))
@pytest.fixture(scope="session")
def notes_module():
"""The module under test, imported once per session."""
import notes
return notes
@pytest.fixture()
def store(tmp_path) -> Path:
"""A fresh, empty store path inside pytest's own temporary directory."""
return tmp_path / "notes.json"
tests/run_tests.sh (25036 bytes)
#!/usr/bin/env bash
# Tests for the Day 080 lab. Run from the lab directory:
# bash tests/run_tests.sh
#
# A command-line interface is a contract with a SHELL, so most of this suite
# launches the real program as a subprocess and inspects what the shell can
# see: the exit code, standard output, and standard error — separately.
#
# Three checks are the ones worth reading, because they are the ones that
# cannot be faked:
#
# * "a bad date writes to stderr, writes NOTHING to stdout, and exits 2"
# captures the two streams into two different files. A program that
# printed its error with print() would pass a combined-output check and
# fail this one, which is exactly the point;
# * "--dry-run leaves the store byte-identical" hashes the file before and
# after. It is paired with a control that runs the same command WITHOUT
# --dry-run and demands the hash changes, so the check cannot pass by the
# program simply never writing anything;
# * "parse_args works in-process with an explicit argv list" runs the pytest
# suite in tests/, which never starts a process at all.
#
# No network, no clock beyond an explicit --on, deterministic. Exits 0 only if
# every check passes.
set -u
export PYTHONDONTWRITEBYTECODE=1
lab_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
failures=0
checks=0
check() {
local label="$1" ok="$2"
checks=$((checks + 1))
if [ "${ok}" = "yes" ]; then
echo " ok: ${label}"
else
echo " FAIL: ${label}"
failures=$((failures + 1))
fi
}
# Resolve tools: an explicit override, then this lab's .venv, then PATH.
# Fails loudly with install instructions rather than silently skipping.
resolve_tool() {
local tool="$1" override="$2"
if [ -n "${override}" ] && [ -x "${override}" ]; then echo "${override}"; return 0; fi
if [ -x "${lab_dir}/.venv/bin/${tool}" ]; then echo "${lab_dir}/.venv/bin/${tool}"; return 0; fi
if command -v "${tool}" >/dev/null 2>&1; then command -v "${tool}"; return 0; fi
return 1
}
python_bin="$(resolve_tool python3 "${PYTHON:-}")" || {
echo "FAIL: python3 not found on PATH." >&2
exit 1
}
pytest_bin="$(resolve_tool pytest "${PYTEST:-}")" || {
echo "FAIL: pytest not found." >&2
echo " Install it with:" >&2
echo " python3 -m venv .venv" >&2
echo " .venv/bin/pip install -r requirements/requirements.txt" >&2
echo " Or point this suite at an existing pytest: PYTEST=/path/to/pytest bash tests/run_tests.sh" >&2
exit 1
}
work="$(mktemp -d "${TMPDIR:-/tmp}/notes-cli.XXXXXX")"
cleanup() { rm -rf "${work}"; }
trap cleanup EXIT
hash_of() { "${python_bin}" -c "
import hashlib, sys, pathlib
print(hashlib.sha256(pathlib.Path(sys.argv[1]).read_bytes()).hexdigest())
" "$1"; }
echo "Day 080 — A Tool You Would Actually Install"
echo
# ==========================================================================
# The behavioural battery. Everything below runs against whichever copy of
# notes.py it is handed, so the reference and your finished starter are held
# to exactly the same standard.
# ==========================================================================
run_battery() {
local label="$1" program="$2"
local dir="${work}/${label}"
local store="${dir}/store.json"
local out="${dir}/out.txt" err="${dir}/err.txt"
mkdir -p "${dir}"
local notes="${python_bin} ${program}"
# ------------------------------------------------------------------------
# 1. Help output — the documentation most users will ever read
# ------------------------------------------------------------------------
${notes} --help >"${out}" 2>"${err}"
local rc=$?
[ "${rc}" -eq 0 ] &&
check "${label}: --help exits 0" "yes" ||
check "${label}: --help exits 0 (got ${rc})" "no"
[ -s "${out}" ] &&
check "${label}: --help writes to standard output" "yes" ||
check "${label}: --help writes to standard output" "no"
[ ! -s "${err}" ] &&
check "${label}: --help writes nothing to standard error" "yes" ||
check "${label}: --help writes nothing to standard error" "no"
local subcommand
for subcommand in add list search export remove; do
grep -q "^ *${subcommand} " "${out}" &&
check "${label}: --help names the '${subcommand}' subcommand" "yes" ||
check "${label}: --help names the '${subcommand}' subcommand" "no"
done
grep -q "usage: notes" "${out}" &&
check "${label}: usage line says 'notes', not the file name (prog=)" "yes" ||
check "${label}: usage line says 'notes', not the file name (prog=)" "no"
grep -q "Exit codes" "${out}" &&
check "${label}: the epilog survives with its line breaks intact" "yes" ||
check "${label}: the epilog survives with its line breaks intact" "no"
${notes} add --help >"${out}" 2>&1
grep -q "YYYY-MM-DD" "${out}" &&
check "${label}: 'add --help' shows the --on metavar" "yes" ||
check "${label}: 'add --help' shows the --on metavar" "no"
grep -q "storage options" "${out}" &&
check "${label}: 'add --help' groups the storage options" "yes" ||
check "${label}: 'add --help' groups the storage options" "no"
${notes} --version >"${out}" 2>&1
rc=$?
{ [ "${rc}" -eq 0 ] && grep -q "^notes 1\.0\.0$" "${out}"; } &&
check "${label}: --version prints 'notes 1.0.0' and exits 0" "yes" ||
check "${label}: --version prints 'notes 1.0.0' and exits 0 (got ${rc})" "no"
# ------------------------------------------------------------------------
# 2. Usage errors — argparse's exit code 2, and nothing on stdout
# ------------------------------------------------------------------------
${notes} >"${out}" 2>"${err}"
rc=$?
[ "${rc}" -eq 2 ] &&
check "${label}: no subcommand at all exits 2" "yes" ||
check "${label}: no subcommand at all exits 2 (got ${rc})" "no"
${notes} frobnicate >"${out}" 2>"${err}"
rc=$?
[ "${rc}" -ne 0 ] &&
check "${label}: an unknown subcommand exits non-zero (${rc})" "yes" ||
check "${label}: an unknown subcommand exits non-zero" "no"
grep -q "invalid choice" "${err}" &&
check "${label}: an unknown subcommand is explained on standard error" "yes" ||
check "${label}: an unknown subcommand is explained on standard error" "no"
${notes} add "x" --tagg typo --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 2 ] && [ ! -s "${out}" ]; } &&
check "${label}: a mistyped option is refused, not ignored" "yes" ||
check "${label}: a mistyped option is refused, not ignored (got ${rc})" "no"
${notes} list -v -q --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 2 ] && grep -q "not allowed with argument" "${err}"; } &&
check "${label}: -v and -q together exit 2, naming both options" "yes" ||
check "${label}: -v and -q together exit 2, naming both options" "no"
${notes} list --format yaml --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 2 ] && grep -q "invalid choice" "${err}"; } &&
check "${label}: a value outside choices= exits 2" "yes" ||
check "${label}: a value outside choices= exits 2" "no"
${notes} list -n 0 --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 2 ] && grep -q "at least 1" "${err}"; } &&
check "${label}: the custom positive_int type rejects 0" "yes" ||
check "${label}: the custom positive_int type rejects 0" "no"
# ---- THE STREAM-SEPARATION CHECK ---------------------------------------
# Two separate files, two separate assertions. A program that printed its
# error message on standard output would pass a 2>&1 check and fail here.
${notes} add "x" --on 2026-13-01 --store "${store}" >"${out}" 2>"${err}"
rc=$?
[ "${rc}" -ne 0 ] &&
check "${label}: a bad date exits non-zero (${rc})" "yes" ||
check "${label}: a bad date exits non-zero" "no"
[ "${rc}" -eq 2 ] &&
check "${label}: a bad date exits exactly 2 — argparse's usage code" "yes" ||
check "${label}: a bad date exits exactly 2 (got ${rc})" "no"
[ ! -s "${out}" ] &&
check "${label}: a bad date writes NOTHING to standard output" "yes" ||
check "${label}: a bad date writes NOTHING to standard output" "no"
[ -s "${err}" ] &&
check "${label}: a bad date writes a message to standard error" "yes" ||
check "${label}: a bad date writes a message to standard error" "no"
grep -q "2026-13-01" "${err}" &&
check "${label}: the message quotes the value that was rejected" "yes" ||
check "${label}: the message quotes the value that was rejected" "no"
grep -q "YYYY-MM-DD" "${err}" &&
check "${label}: the message says what a date should look like" "yes" ||
check "${label}: the message says what a date should look like" "no"
[ ! -e "${store}" ] &&
check "${label}: a refused command created no store file" "yes" ||
check "${label}: a refused command created no store file" "no"
${notes} add "x" --tag "two words" --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 2 ] && grep -q "no spaces or commas" "${err}"; } &&
check "${label}: the custom tag type rejects a tag with a space" "yes" ||
check "${label}: the custom tag type rejects a tag with a space" "no"
# ------------------------------------------------------------------------
# 3. The happy path
# ------------------------------------------------------------------------
${notes} add "ring the dentist" --tag health --on 2026-03-01 --store "${store}" \
>"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 0 ] && grep -q "^added note 1$" "${out}" && [ ! -s "${err}" ]; } &&
check "${label}: a successful add exits 0, reports on stdout, is silent on stderr" "yes" ||
check "${label}: a successful add exits 0, reports on stdout, is silent on stderr (got ${rc})" "no"
${notes} add "argparse turns a script into a tool" -t python -t writing \
--on 2026-03-02 --store "${store}" >/dev/null 2>&1
check "${label}: --tag repeats (action='append')" \
"$(${notes} list --format json --store "${store}" 2>/dev/null |
grep -q '"python"' && echo yes || echo no)"
# ---- stdin, so the tool composes in a pipeline --------------------------
echo "from a pipe" | ${notes} add - --tag inbox --on 2026-03-03 --store "${store}" \
>"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 0 ] && grep -q "^added note 3$" "${out}"; } &&
check "${label}: 'add -' reads the note from a pipe" "yes" ||
check "${label}: 'add -' reads the note from a pipe (got ${rc})" "no"
${notes} search "from a pipe" --store "${store}" >"${out}" 2>&1
grep -q "from a pipe" "${out}" &&
check "${label}: the piped text was really stored" "yes" ||
check "${label}: the piped text was really stored" "no"
# ---- a double dash ends option parsing ---------------------------------
${notes} add --on 2026-03-04 --store "${store}" -- "--not-a-flag" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 0 ] && grep -q "^added note 4$" "${out}"; } &&
check "${label}: '--' lets a note start with two dashes" "yes" ||
check "${label}: '--' lets a note start with two dashes (got ${rc})" "no"
# ---- output formats -----------------------------------------------------
${notes} list --format json --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 0 ] && "${python_bin}" -c "
import json, sys
notes = json.load(open(sys.argv[1]))
assert isinstance(notes, list) and len(notes) == 4, notes
assert notes[0]['text'] == 'ring the dentist', notes[0]
" "${out}"; } &&
check "${label}: 'list --format json' emits parseable JSON on stdout" "yes" ||
check "${label}: 'list --format json' emits parseable JSON on stdout" "no"
${notes} list --store "${store}" >"${out}" 2>&1
grep -q "^ID DATE" "${out}" &&
check "${label}: 'list' defaults to the table format" "yes" ||
check "${label}: 'list' defaults to the table format" "no"
${notes} list -n 1 --store "${store}" >"${out}" 2>&1
[ "$(grep -c '2026-03' "${out}")" -eq 1 ] &&
check "${label}: '-n 1' limits the listing to one note" "yes" ||
check "${label}: '-n 1' limits the listing to one note" "no"
${notes} export --format csv --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 0 ] && head -1 "${out}" | grep -q "^id,date,tags,text$" &&
[ "$(wc -l <"${out}")" -eq 5 ]; } &&
check "${label}: 'export --format csv' writes a header and four rows" "yes" ||
check "${label}: 'export --format csv' writes a header and four rows" "no"
# ---- diagnostics on stderr keep the RESULT on stdout clean --------------
${notes} list --format json -v --store "${store}" >"${out}" 2>"${err}"
{ "${python_bin}" -c "import json,sys; json.load(open(sys.argv[1]))" "${out}" &&
grep -q "4 note(s) from" "${err}"; } &&
check "${label}: -v chatter goes to stderr, leaving stdout valid JSON" "yes" ||
check "${label}: -v chatter goes to stderr, leaving stdout valid JSON" "no"
${notes} add "quiet one" --on 2026-03-05 -q --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 0 ] && [ ! -s "${out}" ] && [ ! -s "${err}" ]; } &&
check "${label}: -q succeeds in silence on both streams" "yes" ||
check "${label}: -q succeeds in silence on both streams" "no"
# ---- search follows grep's exit-code convention -------------------------
${notes} search dentist --store "${store}" >/dev/null 2>&1
rc=$?
[ "${rc}" -eq 0 ] &&
check "${label}: search exits 0 when something matched" "yes" ||
check "${label}: search exits 0 when something matched (got ${rc})" "no"
${notes} search kangaroo --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 1 ] && [ ! -s "${out}" ]; } &&
check "${label}: search exits 1 when nothing matched, printing nothing" "yes" ||
check "${label}: search exits 1 when nothing matched (got ${rc})" "no"
${notes} search DENTIST -i --store "${store}" >/dev/null 2>&1
[ $? -eq 0 ] &&
check "${label}: '-i' matches regardless of case" "yes" ||
check "${label}: '-i' matches regardless of case" "no"
# ------------------------------------------------------------------------
# 4. --dry-run — the promise, and the control that proves it means anything
# ------------------------------------------------------------------------
local before after
before="$(hash_of "${store}")"
${notes} remove 2 --dry-run --store "${store}" >"${out}" 2>"${err}"
rc=$?
after="$(hash_of "${store}")"
[ "${rc}" -eq 0 ] &&
check "${label}: 'remove --dry-run' exits 0" "yes" ||
check "${label}: 'remove --dry-run' exits 0 (got ${rc})" "no"
[ "${before}" = "${after}" ] &&
check "${label}: 'remove --dry-run' leaves the store BYTE-IDENTICAL" "yes" ||
check "${label}: 'remove --dry-run' leaves the store BYTE-IDENTICAL" "no"
grep -q "^would remove note 2$" "${out}" &&
check "${label}: the dry run puts the ids on stdout, where they can be piped" "yes" ||
check "${label}: the dry run puts the ids on stdout, where they can be piped" "no"
grep -q "was not touched" "${err}" &&
check "${label}: the dry run puts its summary on stderr, where it is a diagnostic" "yes" ||
check "${label}: the dry run puts its summary on stderr, where it is a diagnostic" "no"
# A dry run still VALIDATES: an impossible deletion is refused, not promised.
${notes} remove 999 --dry-run --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 1 ] && [ ! -s "${out}" ] && grep -q "no note with id 999" "${err}"; } &&
check "${label}: a dry run still refuses a note that does not exist" "yes" ||
check "${label}: a dry run still refuses a note that does not exist (got ${rc})" "no"
# THE CONTROL: without --dry-run the same command must change the file.
# Without this, "the bytes did not change" could be satisfied by a program
# that never writes at all.
before="$(hash_of "${store}")"
${notes} remove 2 --store "${store}" >"${out}" 2>"${err}"
rc=$?
after="$(hash_of "${store}")"
{ [ "${rc}" -eq 0 ] && [ "${before}" != "${after}" ] &&
grep -q "^removed note 2$" "${out}"; } &&
check "${label}: control — a real remove DOES change the store" "yes" ||
check "${label}: control — a real remove DOES change the store" "no"
${notes} remove 2 --store "${store}" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 1 ] && [ ! -s "${out}" ] && grep -q "no note with id 2" "${err}"; } &&
check "${label}: removing it twice is a refusal on stderr with exit 1" "yes" ||
check "${label}: removing it twice is a refusal on stderr with exit 1 (got ${rc})" "no"
${notes} remove 1 3 --store "${store}" >"${out}" 2>&1
[ "$(grep -c '^removed note' "${out}")" -eq 2 ] &&
check "${label}: nargs='+' removes several notes in one command" "yes" ||
check "${label}: nargs='+' removes several notes in one command" "no"
# ------------------------------------------------------------------------
# 5. Configuration precedence, and composing in a pipeline
# ------------------------------------------------------------------------
local env_store="${dir}/from-env.json"
( cd "${dir}" && NOTES_STORE="${env_store}" ${notes} add "via the environment" \
--on 2026-03-06 >/dev/null 2>&1 )
[ -f "${env_store}" ] &&
check "${label}: \$NOTES_STORE is used when --store is absent" "yes" ||
check "${label}: \$NOTES_STORE is used when --store is absent" "no"
local flag_store="${dir}/from-flag.json"
( cd "${dir}" && NOTES_STORE="${env_store}" ${notes} add "via the flag" \
--on 2026-03-06 --store "${flag_store}" >/dev/null 2>&1 )
{ [ -f "${flag_store}" ] &&
[ "$(${notes} list --format json --store "${env_store}" 2>/dev/null |
grep -c 'via the flag')" -eq 0 ]; } &&
check "${label}: --store beats \$NOTES_STORE (flag wins)" "yes" ||
check "${label}: --store beats \$NOTES_STORE (flag wins)" "no"
# Note the store for this battery is store.json, so ./notes.json appearing
# here can only have come from the built-in default.
( cd "${dir}" && ${notes} add "the built-in default" --on 2026-03-06 >/dev/null 2>&1 )
[ -f "${dir}/notes.json" ] &&
check "${label}: with neither, the built-in ./notes.json default applies" "yes" ||
check "${label}: with neither, the built-in ./notes.json default applies" "no"
# A tool that writes its result to stdout can be piped into another tool.
local piped
piped="$(${notes} export --format json --store "${store}" 2>/dev/null |
"${python_bin}" -c "import json,sys; print(len(json.load(sys.stdin)))")"
[ "${piped}" = "2" ] &&
check "${label}: 'export | python3' composes — stdout is machine-readable" "yes" ||
check "${label}: 'export | python3' composes (got '${piped}')" "no"
# Closing the pipe early must not produce a BrokenPipeError traceback.
${notes} export --format csv --store "${store}" 2>"${err}" | head -1 >/dev/null
[ ! -s "${err}" ] &&
check "${label}: closing the pipe early is not an error" "yes" ||
check "${label}: closing the pipe early is not an error" "no"
# ------------------------------------------------------------------------
# 6. Refusals that are not usage errors exit 1, not 2
# ------------------------------------------------------------------------
echo "{ this is not json" >"${dir}/broken.json"
${notes} list --store "${dir}/broken.json" >"${out}" 2>"${err}"
rc=$?
{ [ "${rc}" -eq 1 ] && [ ! -s "${out}" ] && grep -q "not valid JSON" "${err}"; } &&
check "${label}: a corrupt store is exit 1 on stderr, not a traceback" "yes" ||
check "${label}: a corrupt store is exit 1 on stderr, not a traceback (got ${rc})" "no"
grep -q "Traceback" "${err}" &&
check "${label}: no Python traceback ever reaches the user" "no" ||
check "${label}: no Python traceback ever reaches the user" "yes"
}
# --------------------------------------------------------------------------
echo "1. The reference tool (examples/notes.py)"
# --------------------------------------------------------------------------
run_battery "reference" "${lab_dir}/examples/notes.py"
# --------------------------------------------------------------------------
echo
echo "2. The hand-rolled parser, and why it is not enough"
# --------------------------------------------------------------------------
by_hand_out="${work}/by_hand.txt"
"${python_bin}" "${lab_dir}/examples/by_hand.py" >"${by_hand_out}" 2>&1
by_hand_rc=$?
[ "${by_hand_rc}" -eq 0 ] &&
check "examples/by_hand.py runs and exits 0" "yes" ||
check "examples/by_hand.py runs and exits 0 (got ${by_hand_rc})" "no"
grep -q "8 ordinary command lines, 4 of them handled wrongly." "${by_hand_out}" &&
check "the hand-rolled parser gets 4 of 8 ordinary command lines wrong" "yes" ||
check "the hand-rolled parser gets 4 of 8 ordinary command lines wrong" "no"
grep -q "the tag is silently lost" "${by_hand_out}" &&
check "it loses the value of --tag=shopping without saying so" "yes" ||
check "it loses the value of --tag=shopping without saying so" "no"
grep -q "IndexError" "${by_hand_out}" &&
check "a missing option value gives an IndexError, not a usage message" "yes" ||
check "a missing option value gives an IndexError, not a usage message" "no"
# argparse handles every one of those correctly. Two spot-checks:
ref="${lab_dir}/examples/notes.py"
spot_store="${work}/spot.json"
"${python_bin}" "${ref}" add "buy milk" --tag=shopping --on 2026-03-01 \
--store "${spot_store}" >/dev/null 2>&1
"${python_bin}" "${ref}" list --format json --store "${spot_store}" 2>/dev/null |
grep -q '"shopping"' &&
check "argparse handles --tag=shopping, which the hand-rolled parser dropped" "yes" ||
check "argparse handles --tag=shopping, which the hand-rolled parser dropped" "no"
"${python_bin}" "${ref}" add "x" --tag --store "${spot_store}" >/dev/null 2>"${work}/e.txt"
spot_rc=$?
{ [ "${spot_rc}" -eq 2 ] && grep -q "expected one argument" "${work}/e.txt"; } &&
check "argparse turns a missing option value into a usage message and exit 2" "yes" ||
check "argparse turns a missing option value into a usage message and exit 2" "no"
# --------------------------------------------------------------------------
echo
echo "3. In-process: parse_args with an explicit argv list"
# --------------------------------------------------------------------------
pytest_out="$(cd "${lab_dir}" && NOTES_DIR=examples "${pytest_bin}" tests -q 2>&1)"
pytest_rc=$?
[ "${pytest_rc}" -eq 0 ] &&
check "the in-process pytest suite passes against examples/" "yes" ||
check "the in-process pytest suite passes against examples/ (exit ${pytest_rc})" "no"
if [ "${pytest_rc}" -ne 0 ]; then printf '%s\n' "${pytest_out}" | tail -25; fi
printf '%s' "${pytest_out}" | grep -qE '[0-9]+ passed' &&
check "it reports passing tests ( $(printf '%s' "${pytest_out}" | tail -1) )" "yes" ||
check "it reports passing tests" "no"
# The in-process suite must be testing something. Break the parser's prog name
# in a COPY and demand that the subprocess battery would notice.
sandbox="${work}/sandbox"
mkdir -p "${sandbox}"
cp "${ref}" "${sandbox}/notes.py"
sed -i.bak 's/prog="notes"/prog="notez"/' "${sandbox}/notes.py"
rm -f "${sandbox}/notes.py.bak"
"${python_bin}" "${sandbox}/notes.py" --help 2>&1 | grep -q "usage: notes " &&
check "a broken prog= would be caught (it was not — the check is vacuous)" "no" ||
check "a one-line break to prog= changes the help output, so the check bites" "yes"
# --------------------------------------------------------------------------
echo
echo "4. The starter"
# --------------------------------------------------------------------------
starter="${lab_dir}/starter/notes.py"
"${python_bin}" -c "
import ast, sys
source = open(sys.argv[1], encoding='utf-8').read()
tree = ast.parse(source)
names = {n.name for n in ast.walk(tree) if isinstance(n, ast.FunctionDef)}
required = {
'iso_date', 'positive_int', 'tag_name', 'common_options', 'build_parser',
'parse_args', 'main', 'cmd_add', 'cmd_list', 'cmd_search', 'cmd_export',
'cmd_remove',
}
missing = required - names
assert not missing, f'starter is missing: {sorted(missing)}'
" "${starter}"
[ $? -eq 0 ] &&
check "starter/notes.py is valid Python and defines every required function" "yes" ||
check "starter/notes.py is valid Python and defines every required function" "no"
for n in 1 2 3 4 5 6 7 8; do
grep -q "EXERCISE ${n}" "${starter}" &&
check "starter/notes.py carries EXERCISE ${n}" "yes" ||
check "starter/notes.py carries EXERCISE ${n}" "no"
done
grep -q "argparse" "${starter}" &&
check "starter/notes.py imports argparse" "yes" ||
check "starter/notes.py imports argparse" "no"
remaining="$(grep -c 'raise NotImplementedError' "${starter}" || true)"
if [ "${remaining}" -gt 0 ]; then
echo " .. ${remaining} exercise(s) still unfinished — the behavioural battery"
echo " will run against starter/notes.py once they are done."
else
echo
echo "5. Your finished starter, held to the same standard"
run_battery "starter" "${starter}"
fi
echo
echo "${checks} checks, ${failures} failure(s)."
[ "${failures}" -eq 0 ]
tests/test_parser.py (10515 bytes)
"""In-process tests: the parser and the handlers, with no subprocess at all.
Read this file alongside `run_tests.sh`. The two prove different things:
* run_tests.sh launches the real program and checks exit codes and the two
output streams — the contract a SHELL sees;
* this file calls parse_args(argv) and main(argv, streams) directly — the
contract PYTHON sees.
You need both. A subprocess test is the only honest way to check an exit code
or stream separation. An in-process test is a hundred times faster, gives a
real traceback when it fails, and can assert on the parsed Namespace, which a
subprocess can never see.
That both are possible at all comes down to two design decisions in notes.py:
`parse_args` takes an explicit argument list instead of reading sys.argv, and
`main` takes a Streams object instead of importing sys.stdout. Day 74 called
that injecting the boundary; this is the same idea, one layer out.
"""
from __future__ import annotations
import argparse
import io
import json
import pytest
# ---------------------------------------------------------------------------
# parse_args: the parser as a pure function from a list of strings
# ---------------------------------------------------------------------------
def test_parse_args_takes_an_explicit_list(notes_module):
args = notes_module.parse_args(["add", "hello"])
assert args.command == "add"
assert args.text == "hello"
def test_the_subcommand_binds_its_handler(notes_module):
"""set_defaults(func=...) is the dispatch: no if-statement anywhere."""
for name, handler in [
("add", "cmd_add"),
("list", "cmd_list"),
("search", "cmd_search"),
("export", "cmd_export"),
("remove", "cmd_remove"),
]:
argv = {"add": ["add", "x"], "search": ["search", "x"], "remove": ["remove", "1"]}.get(
name, [name]
)
args = notes_module.parse_args(argv)
assert args.func is getattr(notes_module, handler)
def test_dashes_become_underscores(notes_module):
args = notes_module.parse_args(["remove", "3", "--dry-run"])
assert args.dry_run is True
assert args.ids == [3]
def test_nargs_plus_collects_a_list(notes_module):
args = notes_module.parse_args(["remove", "2", "5", "9"])
assert args.ids == [2, 5, 9]
def test_action_append_collects_repeats(notes_module):
args = notes_module.parse_args(["add", "x", "-t", "one", "--tag", "two"])
assert args.tag == ["one", "two"]
def test_defaults_apply_when_the_option_is_absent(notes_module):
args = notes_module.parse_args(["list"])
assert args.format == "table"
assert args.store is None # None, not "notes.json": see store_path()
assert args.verbose is False and args.quiet is False
def test_type_conversion_happens_during_parsing(notes_module):
from datetime import date
args = notes_module.parse_args(["add", "x", "--on", "2026-03-01"])
assert args.on == date(2026, 3, 1)
assert isinstance(args.on, date)
def test_a_double_dash_ends_option_parsing(notes_module):
args = notes_module.parse_args(["add", "--", "--not-a-flag"])
assert args.text == "--not-a-flag"
@pytest.mark.parametrize(
"argv",
[
[], # no subcommand at all
["frobnicate"], # unknown subcommand
["add"], # missing required positional
["add", "x", "--on", "2026-13-01"], # custom type rejects the value
["add", "x", "--tag", "two words"], # custom type rejects the value
["list", "--format", "yaml"], # not in choices
["list", "-n", "0"], # positive_int rejects it
["list", "-v", "-q"], # mutually exclusive
["remove"], # nargs="+" needs at least one
["add", "x", "--tagg", "typo"], # unknown option is refused, not ignored
],
)
def test_usage_errors_raise_systemexit_with_code_2(notes_module, argv):
"""argparse's contract: a usage error is SystemExit(2), never a traceback."""
with pytest.raises(SystemExit) as caught:
notes_module.parse_args(argv)
assert caught.value.code == 2
def test_help_and_version_exit_zero(notes_module):
for argv in (["--help"], ["--version"], ["add", "--help"]):
with pytest.raises(SystemExit) as caught:
notes_module.parse_args(argv)
assert caught.value.code == 0
# ---------------------------------------------------------------------------
# The custom types, tested on their own — they are just functions
# ---------------------------------------------------------------------------
def test_iso_date_converts(notes_module):
from datetime import date
assert notes_module.iso_date("2026-03-01") == date(2026, 3, 1)
@pytest.mark.parametrize("bad", ["2026-13-01", "tomorrow", "01-03-2026", "", "2026-02-30"])
def test_iso_date_raises_argument_type_error(notes_module, bad):
with pytest.raises(argparse.ArgumentTypeError):
notes_module.iso_date(bad)
@pytest.mark.parametrize("bad", ["0", "-2", "abc", "1.5"])
def test_positive_int_raises_argument_type_error(notes_module, bad):
with pytest.raises(argparse.ArgumentTypeError):
notes_module.positive_int(bad)
def test_the_error_message_names_the_offending_value(notes_module):
with pytest.raises(argparse.ArgumentTypeError) as caught:
notes_module.iso_date("2026-13-01")
assert "2026-13-01" in str(caught.value)
# ---------------------------------------------------------------------------
# main() driven with injected streams — no process, no real terminal
# ---------------------------------------------------------------------------
def run(notes_module, argv, stdin_text=""):
"""Call main() with string streams and return (code, stdout, stderr)."""
streams = notes_module.Streams(
stdin=io.StringIO(stdin_text), stdout=io.StringIO(), stderr=io.StringIO()
)
code = notes_module.main(argv, streams)
return code, streams.stdout.getvalue(), streams.stderr.getvalue()
def test_add_then_list_round_trips(notes_module, store):
code, out, err = run(notes_module, ["add", "buy milk", "--on", "2026-03-01", "--store", str(store)])
assert (code, out, err) == (0, "added note 1\n", "")
code, out, err = run(notes_module, ["list", "--format", "json", "--store", str(store)])
assert code == 0
assert json.loads(out) == [{"date": "2026-03-01", "id": 1, "tags": [], "text": "buy milk"}]
def test_stdin_is_read_when_the_text_is_a_dash(notes_module, store):
code, out, _ = run(
notes_module,
["add", "-", "--on", "2026-03-02", "--store", str(store)],
stdin_text="piped in\n",
)
assert code == 0 and out == "added note 1\n"
assert json.loads(store.read_text())["notes"][0]["text"] == "piped in"
def test_results_go_to_stdout_and_diagnostics_to_stderr(notes_module, store):
run(notes_module, ["add", "one", "--on", "2026-03-01", "--store", str(store)])
code, out, err = run(notes_module, ["list", "--format", "json", "-v", "--store", str(store)])
assert code == 0
assert json.loads(out) # the RESULT parses as JSON, undisturbed...
assert "1 note(s) from" in err # ...because the chatter went elsewhere
def test_quiet_silences_success_but_not_the_work(notes_module, store):
code, out, err = run(
notes_module, ["add", "silent", "--on", "2026-03-01", "-q", "--store", str(store)]
)
assert (code, out, err) == (0, "", "")
assert json.loads(store.read_text())["notes"][0]["text"] == "silent"
def test_a_refusal_exits_1_and_says_so_on_stderr(notes_module, store):
store.write_text('{"version": 1, "notes": []}\n')
code, out, err = run(notes_module, ["remove", "99", "--store", str(store)])
assert code == 1
assert out == ""
assert "no note with id 99" in err
def test_search_exits_1_when_nothing_matched(notes_module, store):
run(notes_module, ["add", "hello", "--on", "2026-03-01", "--store", str(store)])
assert run(notes_module, ["search", "hello", "--store", str(store)])[0] == 0
assert run(notes_module, ["search", "kangaroo", "--store", str(store)])[0] == 1
def test_ignore_case_changes_the_match(notes_module, store):
run(notes_module, ["add", "Hello", "--on", "2026-03-01", "--store", str(store)])
assert run(notes_module, ["search", "hello", "--store", str(store)])[0] == 1
assert run(notes_module, ["search", "hello", "-i", "--store", str(store)])[0] == 0
def test_dry_run_leaves_the_store_byte_identical(notes_module, store):
run(notes_module, ["add", "keep me", "--on", "2026-03-01", "--store", str(store)])
before = store.read_bytes()
code, out, err = run(notes_module, ["remove", "1", "--dry-run", "--store", str(store)])
assert code == 0
assert out == "would remove note 1\n" # the RESULT: pipeable ids
assert "was not touched" in err # the DIAGNOSTIC
assert store.read_bytes() == before # the promise, actually checked
def test_a_real_remove_does_change_the_store(notes_module, store):
"""The control for the test above: without --dry-run, the bytes move."""
run(notes_module, ["add", "keep me", "--on", "2026-03-01", "--store", str(store)])
before = store.read_bytes()
assert run(notes_module, ["remove", "1", "--store", str(store)])[0] == 0
assert store.read_bytes() != before
def test_store_precedence_flag_beats_environment(notes_module, tmp_path, monkeypatch):
from_flag = tmp_path / "flag.json"
from_env = tmp_path / "env.json"
monkeypatch.setenv("NOTES_STORE", str(from_env))
run(notes_module, ["add", "a", "--on", "2026-03-01", "--store", str(from_flag)])
assert from_flag.exists() and not from_env.exists()
run(notes_module, ["add", "b", "--on", "2026-03-01"])
assert from_env.exists()
def test_export_csv_has_a_header_and_one_row_per_note(notes_module, store):
run(notes_module, ["add", "one", "-t", "x", "--on", "2026-03-01", "--store", str(store)])
run(notes_module, ["add", "two", "--on", "2026-03-02", "--store", str(store)])
code, out, _ = run(notes_module, ["export", "--format", "csv", "--store", str(store)])
assert code == 0
assert out.splitlines() == [
"id,date,tags,text",
"1,2026-03-01,x,one",
"2,2026-03-02,,two",
]
def test_a_corrupt_store_is_a_refusal_not_a_traceback(notes_module, store):
store.write_text("{ this is not json")
code, out, err = run(notes_module, ["list", "--store", str(store)])
assert code == 1 and out == "" and "not valid JSON" in err
Troubleshooting
Troubleshooting — Day 080 lab
NotImplementedError: EXERCISE 3: add the storage group ...
Working as designed. Each unfinished exercise raises this the moment its code would have run, so you always know exactly which one you are on. The test suite counts the remaining ones and tells you:
.. 6 exercise(s) still unfinished — the behavioural battery
will run against starter/notes.py once they are done.
Six raises cover eight exercises, because exercises 4, 5 and 6 all live inside
build_parser and share one.
AttributeError: 'Namespace' object has no attribute 'func'
You forgot set_defaults(func=...) on a subparser, or the user ran a command
with no subcommand at all and your add_subparsers call is missing
required=True.
The second case is the interesting one. Without required=True, running plain
notes parses successfully — argparse is happy, there simply is no
subcommand — and then args.func explodes. With it, argparse refuses the
command line, prints usage on standard error, and exits 2, which is what a
user deserves. One keyword turns a crash into a usage message.
argparse.ArgumentError: argument -h/--help: conflicting option string
Your parent parser was built without add_help=False. A parser gets -h by
default; when it is used as a parent, the child inherits that -h and then
tries to add its own. Fix it in one place:
parent = argparse.ArgumentParser(add_help=False)
error: unrecognized arguments: --store /some/path
You put a shared option before the subcommand — notes --store x add "hi" —
but the option is defined on the parent, which the subparsers inherit, not
on the root parser. In this lab's design, shared options go after the
subcommand: notes add "hi" --store x.
This is a real design decision, not a bug in argparse, and both conventions
exist in the wild (git --no-pager log versus docker run --rm). Pick one
and be consistent. If you want an option to work in both positions, add it to
the root parser and the parent — and be aware that the later one wins, which
surprises people.
The --on value arrives as a string, not a date
You passed type=iso_date as a string: type="iso_date". It must be the
function object itself, with no quotes and no call parentheses:
add.add_argument("--on", type=iso_date) # right
add.add_argument("--on", type=iso_date()) # wrong: calls it immediately
add.add_argument("--on", type="iso_date") # wrong: argparse cannot use a string
My error message shows a full Python traceback
You raised ValueError (or let one escape) instead of
argparse.ArgumentTypeError. Only ArgumentTypeError — and TypeError and
ValueError raised inside a type= callable — get converted into a usage
message. An exception raised in your handler is not converted at all: catch
it and turn it into a NotesError, which main() already renders as one
tidy line on standard error with exit 1.
notes add - just sits there doing nothing
Standard input is a terminal and the program is waiting for you to type. Press Ctrl-D (on a line of its own) to signal end-of-file, or Ctrl-C to give up.
That hang is exactly what exercise 7's terminal detection prevents. Once it is
implemented, notes add - with nothing piped in exits 2 with a message
instead of hanging. Try both:
notes add - # detected: exits 2 with an explanation
echo "piped in" | notes add - # reads from the pipe, exits 0
--dry-run prints the right thing but the test still fails
Read the failing check's name. There are two different ones:
- "leaves the store BYTE-IDENTICAL" — your code fell through to
save_store. A dry run mustreturn 0before the write. Re-indent, or add the missingreturn. - "puts the ids on stdout, where they can be piped" — you wrote both lines to the same stream. The ids are a result (standard output); the summary is a diagnostic (standard error).
The dry-run and real-remove output arrive in the wrong order
You wrote to standard output and standard error without flushing in between.
When standard output is a terminal Python line-buffers it, so the order looks
right; the moment you redirect to a file it becomes block-buffered and the
unbuffered standard error overtakes it. Call streams.stdout.flush() before
writing the diagnostic. This is not cosmetic — it is why captured logs from
build servers so often look scrambled.
pytest: command not found
The suite tells you what to do, but for completeness:
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
Or point it at an installation you already have:
PYTEST=/path/to/pytest bash tests/run_tests.sh
ModuleNotFoundError: No module named 'notes' from pytest
tests/conftest.py puts the right directory on sys.path, chosen by the
NOTES_DIR environment variable. Run pytest from the lab directory, not
from inside tests/:
cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
NOTES_DIR=examples pytest tests -q # the reference
NOTES_DIR=starter pytest tests -q # your work
notes list --format json | head -2 prints a BrokenPipeError
head closes the pipe as soon as it has its two lines, and the writer finds
out the hard way. main() already catches BrokenPipeError and returns 0,
because a downstream program deciding it has seen enough is not your program's
failure. If you are writing your own tool and see this, catch it in the same
place — do not sprinkle try/except through the renderers.
Everything passes but the counts differ from expected-output/test-run.txt
test-run.txt was captured with the starter unfinished: 76 checks. With
all eight exercises done the battery runs a second time against your file and
the total is 133 checks. Both are correct; only 0 failure(s). matters.
Security notes
Security notes — Day 080 lab
A command-line tool is a program that takes input from strangers. Sometimes the stranger is you in six months; sometimes it is a shell script running as a scheduled job with arguments assembled from somewhere else. The interface is a trust boundary, and argparse is the first place you can defend it.
What this lab does, and does not do
- No network. Nothing here opens a socket, resolves a name, or downloads anything. The only install is pytest, and that is optional if you already have it.
- No credentials, no keys, no accounts. The tool stores plain notes in a plain JSON file.
- Writes only where you point it. The store path comes from
--store, then$NOTES_STORE, then./notes.jsonin the current directory. The test suite writes exclusively inside amktemp -ddirectory and removes it in atrap, so a failed run leaves nothing behind.
Validation at the boundary is a security control
Every type= function in this tool is a gate. iso_date means no code past
the parser has to wonder whether args.on is a date — it is a
datetime.date or the program already exited 2. positive_int means no
handler receives a negative note id. choices=["table", "json"] means the
format selector can never hold a third value.
This matters more than it looks. The classic command-line vulnerability is a
value that is accepted at the edge, carried untouched through three layers,
and finally interpolated somewhere dangerous. Converting and validating at the
parser — the moment the value enters — means the dangerous shape never exists
inside the program at all. It is the same argument Day 70 made for validating
in __post_init__, moved out to the process boundary.
Never build a shell command out of user input
This tool does not run subprocesses, but the day you write one that does, the rule is absolute:
subprocess.run(["grep", pattern, path]) # a list: safe
subprocess.run(f"grep {pattern} {path}", shell=True) # never do this
With shell=True a pattern of x; rm -rf ~ is not a pattern, it is two
commands. Passing a list means the operating system hands your arguments to
the program directly and no shell ever sees them. Day 81 uses subprocess
properly; this is the habit to bring to it.
Path arguments deserve suspicion
--store takes a path, and a path from a stranger can point anywhere:
../../.ssh/config, a symbolic link, a device file. This lab's tool is a
personal note keeper, so it does what a personal tool should — writes where
you say. A tool that accepts a path from a less trusted source should
additionally:
- resolve it with
Path(value).resolve()and confirm it is inside an expected directory before opening it; - refuse symbolic links where they are not wanted (
Path.is_symlink()); - open with the narrowest mode that works, and set restrictive permissions on files it creates.
--dry-run is a safety feature, not a convenience
Any command that deletes, overwrites, sends, or spends should have one, and it
should be exercised in tests the way this lab's is — by hashing the target
before and after. A --dry-run that has never been checked is a claim, not a
guarantee, and it is the kind of claim people trust right up until the moment
it turns out to be false.
Consider also a confirmation prompt for genuinely destructive operations, with
a --yes/--force flag to skip it in scripts. The pattern is: interactive
users get a chance to stop; automation opts out explicitly and on the record.
Secrets do not belong on the command line
A password or API key passed as --token abc123 is visible to every user on
the machine through the process list, and lands in your shell history file in
plain text. The conventions that avoid this:
- read the secret from an environment variable (
os.environ["API_TOKEN"]); - read it from a file whose path is the argument, not the secret itself
(
--token-file); - read it from standard input, which is exactly the
-convention this lab implements for note text.
The add - exercise is therefore not only about pipelines. It is the same
mechanism you use to keep a secret off the command line.
JSON, not pickle
The store is read with json.loads, which builds only lists, dicts, strings,
numbers, booleans and null — it cannot execute anything. pickle can and does
execute arbitrary code while loading, so a pickle file is as dangerous as a
program. A malformed store here produces notes: ... is not valid JSON and
exit 1; a malicious pickle would produce whatever its author wanted.
The tool also checks the shape of what it loaded, not just that it parsed, and refuses a file that is valid JSON but not a notes store. Parsing and validating are two different steps.
Errors on standard error, always
Sending diagnostics to standard error is a correctness rule, but it has a
security edge: a pipeline like notes export --format json | some-importer
must never have an error message injected into the middle of its data. Keeping
the two streams separate means a downstream program sees either good data or
no data, and never a plausible-looking mixture of the two.
Personal data
Notes are personal data. This store is a plain file in your working directory
with no encryption; treat it accordingly, and keep it out of version control.
A .gitignore entry for notes.json costs nothing.