Programming with PythonPython for Automation and the Web › Day 80

Hands-on lab — Day 80: Building CLIs with argparse

Commands

Setup

cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
python3 --version
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt

Run

python3 examples/by_hand.py
python3 examples/notes.py --help
python3 examples/notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
echo 'from a pipe' | python3 examples/notes.py add - --tag inbox --on 2026-03-03 --store notes.json
python3 examples/notes.py list --format json --store notes.json
python3 examples/notes.py remove 2 --dry-run --store notes.json
python3 starter/notes.py --help

Test

bash tests/run_tests.sh

File tree

examples/by_hand.py
examples/notes.py
expected-output/FIELDS.md
expected-output/help-output.txt
expected-output/sample-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/README.md
requirements/requirements.txt
security.md
starter/notes.py
tests/conftest.py
tests/run_tests.sh
tests/test_parser.py
troubleshooting.md

Lab README

Day 080 lab — A Tool You Would Actually Install

Lesson

Purpose

On Day 56 you built a command-line tool by reading sys.argv yourself. It worked, and for what it did it was the right choice. This lab starts by showing you, in eight ordinary command lines, exactly where that approach stops working — and then builds the thing that replaces it.

You will finish with notes: a real multi-subcommand tool with add, list, search, export and remove, over a small JSON store. Not a toy. It has the things users expect and notice the absence of — -h, --help, --version, short and long option forms, -- to end option parsing, tab-free help text organised into groups, sensible exit codes — and the things that separate a tool from a script: values converted and validated at the parser rather than three functions later, --dry-run on the one destructive command, results on standard output and diagnostics on standard error so it composes in a pipeline, a - argument that reads the note from standard input, and configuration that follows the precedence users already expect (flag, then environment, then default).

The single most useful technique here is subcommands with add_subparsers and set_defaults(func=...). It is how git, docker, pip, and every other tool with more than one verb is structured, and it removes every if command == "add" from your program.

The test suite is unusual and worth reading before you run it: most of it launches your program as a subprocess and inspects what a shell can see — the exit code, standard output, and standard error, captured to separate files. That is the only honest way to test a command-line interface, and it is why one check can prove your streams are genuinely separated rather than merely looking right in a terminal.

Learning objectives

  • Recognise the point at which hand-parsing sys.argv starts producing wrong answers silently, using a working demonstration rather than an assertion.
  • Build an argparse.ArgumentParser with prog, description, epilog and per-argument help, and treat the resulting --help output as a deliverable rather than a side effect.
  • Use add_subparsers with set_defaults(func=...) so dispatch is args.func(args, streams) and no branch on the command name exists anywhere.
  • Write custom type= callables that convert and validate, raising argparse.ArgumentTypeError so a bad value becomes a usage message and exit 2 instead of a traceback.
  • Use choices, default, nargs="+", action="store_true", action="append", argument groups, and a mutually exclusive group, and say what each one buys.
  • Separate results (standard output) from diagnostics (standard error), and prove the separation with a test that captures the two streams apart.
  • Read a note from standard input when the argument is -, and detect a terminal so the tool explains itself instead of hanging.
  • Implement --dry-run as a real guarantee — validated for real, written never — and verify it by hashing the store before and after.
  • Resolve configuration by the expected precedence: flag, then environment, then built-in default.
  • Make the parser testable by having parse_args(argv) take an explicit list, and test both in-process and as a subprocess.

Prerequisites

  • The Day 80 lesson.
  • Day 56: building a data-driven CLI with sys.argv — this lab is its sequel.
  • Days 8–14: the command line itself. Pipes, redirection, and exit codes are assumed knowledge here.
  • Days 64–66: reading and writing files, JSON, and exception strategy.
  • Days 71–74: pytest, and Day 74's argument about injecting a boundary rather than reaching for it — parse_args(argv) and the Streams object are that idea applied to the command line.
  • Day 69: type hints on public functions.
  • A text editor, a terminal, and Python 3.

Supported operating systems

  • macOS — fully supported (tested on macOS 26.5.1, Apple Silicon, Python 3.14.0, pytest 9.1.1, bash 3.2.57).
  • Linux — fully supported (any distribution with Python 3 and bash).
  • Windows — use WSL and follow the Linux path. On a native Windows console substitute python for python3, and note that the suite's banner contains an em dash, so a UTF-8 terminal is needed for it to render. Exit codes, stream separation, and the JSON bytes are identical everywhere.

Hardware requirements

Any computer that runs Python 3. The store this lab writes is a few hundred bytes; the whole suite finishes in about a second. No special memory, disk, GPU, or network.

Required software

  • python3 (3.8 or newer; tested on 3.14.0). The tool itself uses only the standard library: argparse, json, csv, os, sys, pathlib, datetime, dataclasses.
  • pytest for the in-process half of the suite — the only dependency, and only for the tests. See requirements/README.md.
  • bash for the test runner (preinstalled on macOS and Linux).

Free and open-source options

Everything here is free and open source: Python and its standard library, bash, and pytest (MIT licence). No account, API key, network access, or purchase is needed at any point.

The lesson's Alternatives section covers the other options for this job — click and typer, both free and open source, and docopt, which is free but much less actively maintained. None of them is required here, and the reason argparse is the right teaching choice is precisely that it needs no install at all.

Installation

cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
python3 --version                                    # confirm Python 3.8+
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt

If you already have pytest somewhere, you can skip the virtual environment and point the runner at it: PYTEST=/path/to/pytest bash tests/run_tests.sh.

File structure

day-080-building-clis-with-argparse/
├── README.md                    ← you are here
├── metadata.yml                 ← machine-readable lab metadata
├── examples/
│   ├── by_hand.py               ← the sys.argv parser, and the 4 of 8 cases it gets wrong
│   └── notes.py                 ← the complete reference tool
├── starter/
│   └── notes.py                 ← YOUR working file: eight numbered exercises
├── tests/
│   ├── run_tests.sh             ← subprocess checks: exit codes and both streams
│   ├── conftest.py              ← puts examples/ or starter/ on sys.path
│   └── test_parser.py           ← in-process checks: parse_args with an explicit list
├── expected-output/
│   ├── sample-run.txt           ← a real captured session
│   ├── help-output.txt          ← every help screen the tool produces
│   ├── test-run.txt             ← a real captured run of the suite
│   └── FIELDS.md                ← the exact contract your tool must satisfy
├── requirements/
│   ├── requirements.txt         ← pytest==9.1.1, for the tests only
│   └── README.md                ← what each dependency is for
├── troubleshooting.md
└── security.md

Running the tool writes notes.json into whatever directory you run it from, unless you say otherwise. It is safe to delete at any time.

How to run

From this directory:

## 1. See why argparse exists. Eight ordinary command lines, four wrong answers.
python3 examples/by_hand.py

## 2. Read the finished tool's user manual — which it generates itself.
python3 examples/notes.py --help
python3 examples/notes.py add --help

## 3. Drive the reference. Note that --on is explicit, so the output is stable.
python3 examples/notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
python3 examples/notes.py add 'argparse turns a script into a tool' -t python -t writing --on 2026-03-02 --store notes.json

## 4. Read a note from a pipe, which is what the '-' convention is for.
echo 'from a pipe' | python3 examples/notes.py add - --tag inbox --on 2026-03-03 --store notes.json

## 5. Results on standard output, so the tool composes.
python3 examples/notes.py list --store notes.json
python3 examples/notes.py list --format json --store notes.json | python3 -m json.tool

## 6. The dry run, and the proof that it wrote nothing.
shasum -a 256 notes.json
python3 examples/notes.py remove 2 --dry-run --store notes.json
shasum -a 256 notes.json

## 7. Three failures, three different exit codes. Check each with `echo $?`.
python3 examples/notes.py add 'x' --on 2026-13-01 --store notes.json ; echo "exit $?"
python3 examples/notes.py frobnicate                                 ; echo "exit $?"
python3 examples/notes.py remove 99 --store notes.json               ; echo "exit $?"

## 8. The stream separation, made visible.
python3 examples/notes.py list --format json -v --store notes.json > result.json 2> chatter.txt
cat result.json
cat chatter.txt

## 9. Your task: exercises 1-8 in starter/notes.py.
python3 starter/notes.py --help

## 10. Check your work.
bash tests/run_tests.sh

What the commands do

  • python3 examples/by_hand.py — runs a hand-rolled sys.argv parser against eight command lines a user might really type. Three work, one works by luck, and four are handled wrongly: --tag=shopping silently loses its value, the short form -t is swallowed as the note text, the typo --drynrun is accepted and ignored, and a missing option value raises IndexError. This is the argument for argparse, run rather than asserted.
  • python3 examples/notes.py --help — prints the tool's manual. Every word of it comes from the same add_argument calls that do the parsing, which is the reason it cannot drift out of date.
  • The add commands — store two notes with tags and explicit dates. --on is given explicitly so that every capture in expected-output/ is reproducible; leave it off and the note is filed under today.
  • echo 'from a pipe' | ... add - — the - convention: read the value from standard input instead of the command line. This is how a tool joins a pipeline, and it is also how you keep a secret out of your shell history.
  • list --format json | python3 -m json.tool — proves the result on standard output is machine-readable, with nothing else mixed into it.
  • The two shasum calls around remove --dry-run — print the same hash. That is what makes --dry-run a promise rather than a claim.
  • The three failing commands — exit 2 (a usage error argparse caught), 2 again (an unknown subcommand), and 1 (a refusal the tool itself understood). Different numbers because a script downstream needs to tell them apart.
  • The redirect in step 8 — result.json holds valid JSON; chatter.txt holds 2 note(s) from notes.json. Two streams, two destinations, no interference.
  • python3 starter/notes.py --help — fails until exercise 4 is done. That is the point: nothing works before the parser exists.
  • bash tests/run_tests.sh — 76 checks while the starter is unfinished, 133 once all eight exercises are complete. Exits 0 only if every check passes.

Expected output

See expected-output/sample-run.txt for the full captured session and expected-output/help-output.txt for every help screen. The heart of it:

$ python3 notes.py list --store notes.json
ID  DATE        TAGS            TEXT
------------------------------------
 1  2026-03-01  health          ring the dentist
 2  2026-03-02  python,writing  argparse turns a script into a tool
 3  2026-03-03  inbox           from a pipe
exit: 0

$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json

$ python3 notes.py remove 2 --dry-run --store notes.json
would remove note 2
dry run: 1 note(s) would be removed; notes.json was not touched
exit: 0

$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json

$ python3 notes.py add 'x' --on 2026-13-01 --store notes.json
usage: notes add [-h] [--store PATH] [-v | -q] [-t TAG] [--on YYYY-MM-DD]
                 [--dry-run]
                 text
notes add: error: argument --on: '2026-13-01' is not a date in YYYY-MM-DD form (for example 2026-03-01)
exit: 2

Every date is supplied explicitly with --on and nothing reads the network or a random number, so these bytes are reproducible on any machine with Python 3. expected-output/FIELDS.md states the full contract: every exit code, which stream each kind of output belongs on, the dry-run guarantee, the precedence rules, and the in-process parser behaviour.

Validation steps

  1. python3 examples/by_hand.py ends with 8 ordinary command lines, 4 of them handled wrongly.
  2. python3 examples/notes.py --help exits 0 and names all five subcommands.
  3. python3 examples/notes.py frobnicate; echo $? prints 2, and the message appears on standard error — confirm with python3 examples/notes.py frobnicate 2>/dev/null, which prints nothing.
  4. python3 examples/notes.py add 'x' --on 2026-13-01 >out.txt 2>err.txt leaves out.txt empty and err.txt non-empty. Check both. This is the single check that proves the streams were separated on purpose.
  5. The two shasum -a 256 notes.json calls around remove --dry-run print the identical hash; the one after a real remove differs.
  6. python3 examples/notes.py search kangaroo; echo $? prints 1 and no output — grep's convention, not an error.
  7. echo 'x' | python3 examples/notes.py add - --on 2026-03-09 --store t.json exits 0 and the note text really is x.
  8. Every exercise in starter/notes.py is done — grep -c 'raise NotImplementedError' starter/notes.py returns 0.
  9. bash tests/run_tests.sh reports 0 failure(s). and exits 0.

Tests

bash tests/run_tests.sh

Expected final line while the starter is unfinished: 76 checks, 0 failure(s). Once all eight exercises are complete the same battery runs a second time against your file, giving 133 checks, 0 failure(s). Both are correct; only the failure count matters. The command exits 0 on success and non-zero on any failure, so it can run in CI. A full captured run is in expected-output/test-run.txt.

The suite has two halves, and the split is the lesson:

  • Subprocess checks launch the real program and inspect exit codes and the two streams separately. Only a subprocess can observe an exit code, and only separate capture can prove stream separation.
  • In-process pytest checks (tests/test_parser.py) call parse_args(["add", "hello"]) and main(argv, streams) directly, with io.StringIO standing in for the terminal. They are far faster, give real tracebacks, and can assert on the parsed namespace — which a subprocess can never see. They are only possible because parse_args takes an explicit list and main takes its streams as a parameter.

Two checks deserve a look before you run them. The dry-run check is paired with a control that runs the same command without --dry-run and demands the hash changes — without it, "the file did not change" would also be satisfied by a tool that never writes at all. And the suite breaks a copy of the reference with sed, then confirms the help check notices, so you know the checks are not vacuous.

Cleanup

The lab writes only notes.json, into whatever directory you ran the tool from, plus result.json and chatter.txt if you followed step 8:

rm -f notes.json result.json chatter.txt t.json

To reset your work: git checkout -- starter/. To remove the virtual environment: rm -rf .venv. The test runner makes its own directory with mktemp -d and removes it in a trap, so even a failed run leaves nothing behind.

Troubleshooting

See troubleshooting.md for the full list: the NotImplementedError from each unfinished exercise, the AttributeError: 'Namespace' object has no attribute 'func' that means a missing set_defaults or a missing required=True, the conflicting option string: -h that means a parent parser without add_help=False, why type=iso_date() with parentheses is wrong, why a traceback means you raised ValueError instead of ArgumentTypeError, why notes add - hangs before exercise 7 is done, why dry-run output can arrive out of order without a flush, and how to point the suite at an existing pytest.

Security notes

See security.md. Short version: the command line is a trust boundary, and every type= callable is a gate on it — a value that is converted and validated at the parser can never exist in a dangerous shape further in. Never build a shell command by string-formatting user input; pass a list and no shell ever sees it. Secrets do not belong in arguments, because arguments are visible in the process list and in shell history — which is the second reason the - stdin convention exists. json.loads cannot execute code; pickle can. And a --dry-run that has never been tested is a claim, not a guarantee.

Extension exercises

  1. Add an edit subcommand. It should take an id and either new text as an argument or - to read from standard input. Count how many places you had to touch. If it was more than one add_parser block and one handler, something in your design is not carrying its weight.
  2. Add a config file. Read defaults from ~/.notesrc (a small JSON file) and slot it into the precedence chain between the built-in defaults and the environment: defaults, then config file, then $NOTES_STORE, then the flag. Use parser.set_defaults(**from_config) — argparse supports exactly this, and doing it any other way means writing the precedence logic by hand.
  3. Add shell completion, honestly. argparse has no built-in completion. Write a small bash completion function by hand that offers the five subcommand names, and note in a comment which third-party package (argcomplete) would generate it for you and what that would cost in dependencies. Deciding not to add a dependency is a real engineering act.
  4. Make the destructive path safer. Add an interactive confirmation to remove, plus --yes to skip it. Then work out how to test the prompt — the answer is that Streams already gives you the seam.
  5. Break the streams on purpose. Change one streams.stderr.write in cmd_remove to streams.stdout.write and run the suite. Watch exactly one check fail and read its name. Then put it back. Knowing which check catches a mistake is worth more than being told the rule.
  • Previous day: Day 79 — Web Scraping Basics (labs/sections/programming-with-python/day-079-web-scraping-basics/).
  • Next day: Day 81 — Scheduling and Background Jobs (labs/sections/programming-with-python/day-081-scheduling-and-background-jobs/), which schedules a tool like this one and needs its exit codes to be right.
  • Week 12 project: the Personal Automation Toolkit (labs/sections/programming-with-python/projects/week-12/). The notes tool is a direct rehearsal for its command-line front end.

Expected output

FIELDS.md

# Expected output — Day 080 lab

These are real captured runs from the authoring machine (macOS 26.5.1, Apple
Silicon, Python 3.14.0, pytest 9.1.1, bash 3.2.57, 2026-07-19). Every date the
tool records comes from an explicit `--on`, and nothing here reads the
network or a random number, so the same commands produce the same bytes on
any machine with Python 3.

## Files

- `sample-run.txt` — a full session with the reference tool: adding notes by
  argument and through a pipe, listing as a table and as JSON, searching,
  exporting CSV, a `--dry-run` with the store's SHA-256 hash printed before
  and after, the real removal, the two output streams redirected to separate
  files, and five different failures with their exit codes.
- `help-output.txt` — every help screen the tool produces: the root `--help`,
  all five subcommand `--help` screens, and `--version`. Read this one as a
  document, not as output: it is the tool's entire user manual, generated
  from the same declarations that do the parsing.
- `test-run.txt` — a full run of `bash tests/run_tests.sh` with the starter
  exercises unfinished: 76 checks, 0 failures, exit 0. One line in it varies
  between runs — the check that reports `42 passed in 0.07s` quotes pytest's
  own timing, which is a measurement and will differ by a few hundredths of a
  second on your machine. Every other line is byte-stable.

## The contract your finished tool must satisfy

### Exit codes

| Command | Exit code | Why |
| --- | --- | --- |
| `notes --help`, `notes add --help`, `notes --version` | 0 | asking for help is not an error |
| a command that did its job | 0 | the only success code |
| `notes search kangaroo` with no match | 1 | grep's convention: nothing found, nothing wrong |
| `notes remove 99` when 99 does not exist | 1 | a refusal the tool understood |
| a store that is not valid JSON | 1 | a refusal, not a traceback |
| `notes` with no subcommand | 2 | usage error |
| `notes frobnicate` | 2 | usage error |
| `notes add x --on 2026-13-01` | 2 | the custom `type=` rejected the value |
| `notes list --format yaml` | 2 | outside `choices=` |
| `notes list -v -q` | 2 | mutually exclusive options |
| `notes add x --tagg typo` | 2 | an unknown option is refused, never ignored |

### Which stream each thing goes to

| Output | Stream | Reason |
| --- | --- | --- |
| the note ids from `remove --dry-run` | standard output | it is a RESULT; a person can pipe it |
| `list`, `search`, `export` output | standard output | the result again |
| `added note 3`, `removed note 2` | standard output | a short confirmation the user asked for |
| every `-v` explanation | standard error | a diagnostic; it must not pollute a pipe |
| the `dry run: ... was not touched` summary | standard error | a diagnostic |
| every error message | standard error | so `2>/dev/null` still leaves the result usable |
| argparse's own usage errors | standard error | argparse does this for you, and it is right |

The single check that proves this was implemented rather than approximated:
`notes add x --on 2026-13-01 >out.txt 2>err.txt` must leave `out.txt` **empty**
and `err.txt` **non-empty**. A program that printed its errors with a bare
`print()` passes a `2>&1` check and fails this one.

### The dry-run promise

| Step | Required |
| --- | --- |
| `shasum -a 256 notes.json` before | some hash H |
| `notes remove 2 --dry-run` | exits 0, prints `would remove note 2` on stdout |
| `shasum -a 256 notes.json` after | **exactly H** |
| `notes remove 99 --dry-run` | still exits 1 — a dry run validates for real |
| `notes remove 2` (no `--dry-run`) | exits 0, and the hash **changes** |

That last row is the control. Without it, "the file did not change" would also
be satisfied by a tool that never writes anything.

### Configuration precedence

| Command | Store used |
| --- | --- |
| `notes add x --store a.json` with `NOTES_STORE=b.json` set | `a.json` — the flag wins |
| `notes add x` with `NOTES_STORE=b.json` set | `b.json` — the environment |
| `notes add x` with nothing set | `./notes.json` — the built-in default |

### Parser behaviour, in-process

| Call | Result |
| --- | --- |
| `parse_args(["add", "hello"])` | `args.command == "add"`, `args.text == "hello"` |
| `parse_args(["remove", "3", "--dry-run"])` | `args.dry_run is True`, `args.ids == [3]` |
| `parse_args(["remove", "2", "5", "9"])` | `args.ids == [2, 5, 9]` — `nargs="+"` |
| `parse_args(["add", "x", "-t", "a", "--tag", "b"])` | `args.tag == ["a", "b"]` — `action="append"` |
| `parse_args(["add", "x", "--on", "2026-03-01"])` | `args.on` is a `datetime.date`, already converted |
| `parse_args(["add", "--", "--not-a-flag"])` | `args.text == "--not-a-flag"` |
| `parse_args(["list"]).store` | `None` — not `"notes.json"`; see `store_path()` |
| `parse_args(["list"]).func` | `is cmd_list` — the `set_defaults` dispatch |
| any usage error | raises `SystemExit` with `code == 2` |
| `parse_args(["--help"])` | raises `SystemExit` with `code == 0` |
| `iso_date("2026-13-01")` | raises `argparse.ArgumentTypeError` naming the value |
| `positive_int("0")` | raises `argparse.ArgumentTypeError` saying "at least 1" |

## Platform notes

- macOS and Linux produce identical bytes. The only thing that differs
  between the two in `sample-run.txt` is the temporary directory the capture
  ran in, and that never appears in the output.
- `shasum -a 256` is the macOS spelling. On most Linux distributions the
  command is `sha256sum`; the hash is the same either way. The test suite
  does not depend on either — it hashes the file with Python's `hashlib`, so
  it works everywhere.
- **Colour.** From Python 3.13 argparse colourizes its help and usage output
  when it detects a terminal. Every capture in this directory was made with
  output redirected to a file, so no colour codes appear in them. Run the same
  commands interactively on a recent Python and the *text* is identical while
  the terminal shows it in colour. Nothing in the test suite depends on
  colour, because everything it inspects is captured to a file.
- On Windows, run everything inside WSL. Two things genuinely differ on a
  native Windows console: `python3` may be spelled `python`, and the em dash
  in the suite's banner needs a UTF-8 terminal. Exit codes, stream
  separation and the JSON bytes are unaffected.
- The suite writes only inside a directory made with `mktemp -d` and removes
  it in a `trap`, so even a failed run leaves nothing behind.

help-output.txt

# Every help screen this tool produces, captured from real runs.
# Help output is the documentation most users will ever read, so it is
# worth looking at all of it in one place.

$ python3 notes.py --help
usage: notes [-h] [--version] <command> ...

Keep short notes in a JSON file you can read with your own eyes.

options:
  -h, --help  show this help message and exit
  --version   print the version and exit

subcommands:
  <command>
    add       add one note
    list      list notes
    search    find notes containing text
    export    write every note to standard output
    remove    delete notes by id

Examples:
  notes add 'ring the dentist' --tag health --on 2026-03-01
  echo 'from a pipe' | notes add - --tag inbox
  notes list --format json | python3 -m json.tool
  notes remove 3 --dry-run

Exit codes: 0 success, 1 refusal, 2 usage error.
exit: 0

$ python3 notes.py add --help
usage: notes add [-h] [--store PATH] [-v | -q] [-t TAG] [--on YYYY-MM-DD]
                 [--dry-run]
                 text

Add one note. Use '-' as the text to read it from standard input.

positional arguments:
  text             the note text, or '-' to read standard input

options:
  -h, --help       show this help message and exit
  -t, --tag TAG    attach a tag; repeat the option for more than one
  --on YYYY-MM-DD  the date to file the note under (default: today)
  --dry-run        say what would happen without writing anything

storage options:
  where the notes live. Precedence: --store, then $NOTES_STORE, then
  ./notes.json

  --store PATH     path to the JSON store (default: $NOTES_STORE or
                   ./notes.json)

output options:
  -v, --verbose    explain what is happening, on standard error
  -q, --quiet      print nothing on success
exit: 0

$ python3 notes.py list --help
usage: notes list [-h] [--store PATH] [-v | -q] [--format {table,json}]
                  [-t TAG] [--since YYYY-MM-DD] [-n N]

List notes, newest last.

options:
  -h, --help            show this help message and exit
  --format {table,json}
                        output format (default: table)
  -t, --tag TAG         only notes with this tag
  --since YYYY-MM-DD    only notes on or after this date
  -n, --limit N         show at most N notes

storage options:
  where the notes live. Precedence: --store, then $NOTES_STORE, then
  ./notes.json

  --store PATH          path to the JSON store (default: $NOTES_STORE or
                        ./notes.json)

output options:
  -v, --verbose         explain what is happening, on standard error
  -q, --quiet           print nothing on success
exit: 0

$ python3 notes.py search --help
usage: notes search [-h] [--store PATH] [-v | -q] [-i] [--format {table,json}]
                    pattern

Find notes whose text contains PATTERN. Exits 1 when nothing matched.

positional arguments:
  pattern               the text to look for

options:
  -h, --help            show this help message and exit
  -i, --ignore-case     match regardless of case
  --format {table,json}
                        output format (default: table)

storage options:
  where the notes live. Precedence: --store, then $NOTES_STORE, then
  ./notes.json

  --store PATH          path to the JSON store (default: $NOTES_STORE or
                        ./notes.json)

output options:
  -v, --verbose         explain what is happening, on standard error
  -q, --quiet           print nothing on success
exit: 0

$ python3 notes.py export --help
usage: notes export [-h] [--store PATH] [-v | -q] [--format {json,csv}]

Write every note to standard output so it can be piped or redirected.

options:
  -h, --help           show this help message and exit
  --format {json,csv}  output format (default: json)

storage options:
  where the notes live. Precedence: --store, then $NOTES_STORE, then
  ./notes.json

  --store PATH         path to the JSON store (default: $NOTES_STORE or
                       ./notes.json)

output options:
  -v, --verbose        explain what is happening, on standard error
  -q, --quiet          print nothing on success
exit: 0

$ python3 notes.py remove --help
usage: notes remove [-h] [--store PATH] [-v | -q] [--dry-run] ID [ID ...]

Delete one or more notes. Destructive, so it has --dry-run.

positional arguments:
  ID             note id to remove

options:
  -h, --help     show this help message and exit
  --dry-run      list what would be removed and leave the store untouched

storage options:
  where the notes live. Precedence: --store, then $NOTES_STORE, then
  ./notes.json

  --store PATH   path to the JSON store (default: $NOTES_STORE or
                 ./notes.json)

output options:
  -v, --verbose  explain what is happening, on standard error
  -q, --quiet    print nothing on success
exit: 0

$ python3 notes.py --version
notes 1.0.0
exit: 0

sample-run.txt

# A real session with the reference tool. Every line below was captured
# from an actual run; the store lives in a scratch directory.

$ python3 notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
added note 1
exit: 0

$ python3 notes.py add 'argparse turns a script into a tool' -t python -t writing --on 2026-03-02 --store notes.json
added note 2
exit: 0

$ echo 'from a pipe' | python3 notes.py add - --tag inbox --on 2026-03-03 --store notes.json
added note 3
exit: 0

$ python3 notes.py list --store notes.json
ID  DATE        TAGS            TEXT
------------------------------------
 1  2026-03-01  health          ring the dentist
 2  2026-03-02  python,writing  argparse turns a script into a tool
 3  2026-03-03  inbox           from a pipe
exit: 0

$ python3 notes.py list --format json --store notes.json
[
  {
    "date": "2026-03-01",
    "id": 1,
    "tags": [
      "health"
    ],
    "text": "ring the dentist"
  },
  {
    "date": "2026-03-02",
    "id": 2,
    "tags": [
      "python",
      "writing"
    ],
    "text": "argparse turns a script into a tool"
  },
  {
    "date": "2026-03-03",
    "id": 3,
    "tags": [
      "inbox"
    ],
    "text": "from a pipe"
  }
]
exit: 0

$ python3 notes.py search PIPE -i --store notes.json
ID  DATE        TAGS   TEXT
---------------------------
 3  2026-03-03  inbox  from a pipe
exit: 0

$ python3 notes.py search kangaroo --store notes.json
exit: 1

$ python3 notes.py export --format csv --store notes.json
id,date,tags,text
1,2026-03-01,health,ring the dentist
2,2026-03-02,python;writing,argparse turns a script into a tool
3,2026-03-03,inbox,from a pipe
exit: 0

# --- a destructive command, rehearsed first --------------------------

$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json
exit: 0

$ python3 notes.py remove 2 --dry-run --store notes.json
would remove note 2
dry run: 1 note(s) would be removed; notes.json was not touched
exit: 0

$ shasum -a 256 notes.json
c112b7e42cf886f911baa54b62a57b348e501328ddb6ec749f62c5023d70dca0 notes.json
exit: 0

# The two hashes are identical: --dry-run really did write nothing.

$ python3 notes.py remove 2 --store notes.json
removed note 2
exit: 0

$ python3 notes.py list --store notes.json
ID  DATE        TAGS    TEXT
----------------------------
 1  2026-03-01  health  ring the dentist
 3  2026-03-03  inbox   from a pipe
exit: 0

# --- the two streams are separate, which is what makes it composable -

$ python3 notes.py list --format json -v --store notes.json > result.json 2> chatter.txt
exit: 0

$ head -4 result.json      # standard output: the RESULT, still valid JSON
[
  {
    "date": "2026-03-01",
    "id": 1,
exit: 0

$ cat chatter.txt          # standard error: the DIAGNOSTIC
2 note(s) from notes.json
exit: 0

# --- usage errors: exit 2, message on stderr, nothing on stdout ------

$ python3 notes.py add 'x' --on 2026-13-01 --store notes.json
usage: notes add [-h] [--store PATH] [-v | -q] [-t TAG] [--on YYYY-MM-DD]
                 [--dry-run]
                 text
notes add: error: argument --on: '2026-13-01' is not a date in YYYY-MM-DD form (for example 2026-03-01)
exit: 2

$ python3 notes.py add 'x' --on 2026-13-01 --store notes.json 2>/dev/null   # stderr discarded
exit: 2  (and standard output was empty — nothing printed above)

$ python3 notes.py frobnicate --store notes.json
usage: notes [-h] [--version] <command> ...
notes: error: argument <command>: invalid choice: 'frobnicate' (choose from add, list, search, export, remove)
exit: 2

$ python3 notes.py list -v -q --store notes.json
usage: notes list [-h] [--store PATH] [-v | -q] [--format {table,json}]
                  [-t TAG] [--since YYYY-MM-DD] [-n N]
notes list: error: argument -q/--quiet: not allowed with argument -v/--verbose
exit: 2

$ python3 notes.py remove 99 --store notes.json
notes: no note with id 99 in notes.json
exit: 1

test-run.txt

# A real run of the Day 080 test suite on the authoring machine
# (macOS 26.5.1, Apple Silicon, Python 3.14.0, pytest 9.1.1, bash 3.2.57).
# The starter exercises are unfinished here, so the behavioural battery
# runs against the reference only. Finish all eight and it runs twice,
# giving 133 checks.

$ bash tests/run_tests.sh
Day 080 — A Tool You Would Actually Install

1. The reference tool (examples/notes.py)
  ok: reference: --help exits 0
  ok: reference: --help writes to standard output
  ok: reference: --help writes nothing to standard error
  ok: reference: --help names the 'add' subcommand
  ok: reference: --help names the 'list' subcommand
  ok: reference: --help names the 'search' subcommand
  ok: reference: --help names the 'export' subcommand
  ok: reference: --help names the 'remove' subcommand
  ok: reference: usage line says 'notes', not the file name (prog=)
  ok: reference: the epilog survives with its line breaks intact
  ok: reference: 'add --help' shows the --on metavar
  ok: reference: 'add --help' groups the storage options
  ok: reference: --version prints 'notes 1.0.0' and exits 0
  ok: reference: no subcommand at all exits 2
  ok: reference: an unknown subcommand exits non-zero (2)
  ok: reference: an unknown subcommand is explained on standard error
  ok: reference: a mistyped option is refused, not ignored
  ok: reference: -v and -q together exit 2, naming both options
  ok: reference: a value outside choices= exits 2
  ok: reference: the custom positive_int type rejects 0
  ok: reference: a bad date exits non-zero (2)
  ok: reference: a bad date exits exactly 2 — argparse's usage code
  ok: reference: a bad date writes NOTHING to standard output
  ok: reference: a bad date writes a message to standard error
  ok: reference: the message quotes the value that was rejected
  ok: reference: the message says what a date should look like
  ok: reference: a refused command created no store file
  ok: reference: the custom tag type rejects a tag with a space
  ok: reference: a successful add exits 0, reports on stdout, is silent on stderr
  ok: reference: --tag repeats (action='append')
  ok: reference: 'add -' reads the note from a pipe
  ok: reference: the piped text was really stored
  ok: reference: '--' lets a note start with two dashes
  ok: reference: 'list --format json' emits parseable JSON on stdout
  ok: reference: 'list' defaults to the table format
  ok: reference: '-n 1' limits the listing to one note
  ok: reference: 'export --format csv' writes a header and four rows
  ok: reference: -v chatter goes to stderr, leaving stdout valid JSON
  ok: reference: -q succeeds in silence on both streams
  ok: reference: search exits 0 when something matched
  ok: reference: search exits 1 when nothing matched, printing nothing
  ok: reference: '-i' matches regardless of case
  ok: reference: 'remove --dry-run' exits 0
  ok: reference: 'remove --dry-run' leaves the store BYTE-IDENTICAL
  ok: reference: the dry run puts the ids on stdout, where they can be piped
  ok: reference: the dry run puts its summary on stderr, where it is a diagnostic
  ok: reference: a dry run still refuses a note that does not exist
  ok: reference: control — a real remove DOES change the store
  ok: reference: removing it twice is a refusal on stderr with exit 1
  ok: reference: nargs='+' removes several notes in one command
  ok: reference: $NOTES_STORE is used when --store is absent
  ok: reference: --store beats $NOTES_STORE (flag wins)
  ok: reference: with neither, the built-in ./notes.json default applies
  ok: reference: 'export | python3' composes — stdout is machine-readable
  ok: reference: closing the pipe early is not an error
  ok: reference: a corrupt store is exit 1 on stderr, not a traceback
  ok: reference: no Python traceback ever reaches the user

2. The hand-rolled parser, and why it is not enough
  ok: examples/by_hand.py runs and exits 0
  ok: the hand-rolled parser gets 4 of 8 ordinary command lines wrong
  ok: it loses the value of --tag=shopping without saying so
  ok: a missing option value gives an IndexError, not a usage message
  ok: argparse handles --tag=shopping, which the hand-rolled parser dropped
  ok: argparse turns a missing option value into a usage message and exit 2

3. In-process: parse_args with an explicit argv list
  ok: the in-process pytest suite passes against examples/
  ok: it reports passing tests ( 42 passed in 0.07s )
  ok: a one-line break to prog= changes the help output, so the check bites

4. The starter
  ok: starter/notes.py is valid Python and defines every required function
  ok: starter/notes.py carries EXERCISE 1
  ok: starter/notes.py carries EXERCISE 2
  ok: starter/notes.py carries EXERCISE 3
  ok: starter/notes.py carries EXERCISE 4
  ok: starter/notes.py carries EXERCISE 5
  ok: starter/notes.py carries EXERCISE 6
  ok: starter/notes.py carries EXERCISE 7
  ok: starter/notes.py carries EXERCISE 8
  ok: starter/notes.py imports argparse
  ..  6 exercise(s) still unfinished — the behavioural battery
      will run against starter/notes.py once they are done.

76 checks, 0 failure(s).
exit: 0

Source files

examples/by_hand.py (3779 bytes)
#!/usr/bin/env python3
"""by_hand.py — what parsing sys.argv yourself actually costs.

Day 56 built a data-driven command-line tool with sys.argv, and for one
positional argument that is the right call: argparse would have been more
machinery than the job needed. This file shows the exact point where that
stops being true.

Run it with no arguments to watch a hand-rolled parser be put through eight
ordinary command lines, four of which it gets wrong. Nothing here is a straw
man: this is the shape people really write, right down to the manual index
arithmetic.

    python3 by_hand.py

The point is not that hand-parsing is impossible. It is that everything the
hand-rolled version is missing — help text, a usage message, exit code 2,
short and long forms, `--flag=value`, clustered short flags, `--` to end
option parsing, type conversion, and the promise that an unknown option is
refused rather than ignored — is a line of argparse each.
"""

from __future__ import annotations

import sys


def parse_by_hand(argv: list[str]) -> dict[str, object]:
    """A hand-rolled parser for: notes add TEXT [--tag T] [--dry-run].

    Written the way a hand-rolled parser is usually written: walk the list,
    branch on what you find, and hope.
    """
    text: str | None = None
    tag: str | None = None
    dry_run = False

    i = 0
    while i < len(argv):
        item = argv[i]
        if item == "--dry-run":
            dry_run = True
        elif item == "--tag":
            i += 1
            tag = argv[i]  # IndexError if the user forgot the value
        elif text is None:
            text = item
        i += 1

    return {"text": text, "tag": tag, "dry_run": dry_run}


# Each case is (argv, correct?, comment). The `correct?` flag is stated
# explicitly rather than guessed from the text, because a count that depends
# on string-matching its own commentary is exactly the kind of quiet wrongness
# this file is about.
CASES: list[tuple[list[str], bool, str]] = [
    (["buy milk"], True, "the happy path — this one works"),
    (["buy milk", "--tag", "shopping"], True, "an option with a value — also fine"),
    (["buy milk", "--tag", "shopping", "--dry-run"], True, "two options — still fine"),
    (["buy milk", "--tag=shopping"], False, "--flag=value form: the tag is silently lost"),
    (["buy milk", "-t", "shopping"], False, "short form: -t is swallowed as the text"),
    (["buy milk", "--drynrun"], False, "a typo: accepted in silence, and does nothing"),
    (["--dry-run", "buy milk"], True, "options before the text — right answer, by luck"),
    (["buy milk", "--tag"], False, "a missing value: IndexError, and a traceback"),
]


def main() -> int:
    print("A hand-rolled parser for: notes add TEXT [--tag T] [--dry-run]")
    print()
    wrong = 0
    for argv, correct, comment in CASES:
        printable = " ".join(repr(a) if " " in a else a for a in argv)
        try:
            result = parse_by_hand(argv)
            got = f"text={result['text']!r} tag={result['tag']!r} dry_run={result['dry_run']}"
        except IndexError:
            got = "IndexError: list index out of range"
        print(f"  $ notes add {printable}")
        print(f"      {got}")
        print(f"      {comment}")
        print()
        if not correct:
            wrong += 1

    print(f"{len(CASES)} ordinary command lines, {wrong} of them handled wrongly.")
    print()
    print("None of these is exotic. Every one of them is something a user")
    print("types on their first afternoon with your tool. And this parser")
    print("still has no --help, no usage message, no exit code 2, no type")
    print("conversion, and no way to refuse an option it does not know.")
    return 0


if __name__ == "__main__":
    sys.exit(main())
examples/notes.py (18631 bytes)
#!/usr/bin/env python3
"""notes — a small note-taking tool, built the way a real command should be.

This is the reference implementation for the Day 080 lab. Everything in it is
standard library: argparse, json, csv, os, sys, pathlib, datetime.

The shape to notice, because it is the shape of every serious Python CLI:

  build_parser()          builds the parser and NOTHING else — no I/O, no work
  parse_args(argv)        takes an EXPLICIT list, so tests never touch sys.argv
  cmd_add / cmd_list /    one handler per subcommand, bound by set_defaults
  cmd_search / ...        so dispatch is `args.func(args, streams)`
  main(argv, streams)     wires them together and returns an exit code

Streams are injected rather than reached for, so the same code can be driven
by a test with io.StringIO in place of the terminal. That is Day 74's boundary
lesson applied to standard input and standard output.

Exit codes:
  0  the command did what it said
  1  a runtime refusal (no such note, unreadable store)
  2  a usage error (argparse's own convention, and we keep it)
"""

from __future__ import annotations

import argparse
import csv
import json
import os
import sys
from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path
from typing import Any, Callable, Sequence, TextIO

__version__ = "1.0.0"

DEFAULT_STORE = "notes.json"
STORE_ENV_VAR = "NOTES_STORE"


# ---------------------------------------------------------------------------
# Errors
# ---------------------------------------------------------------------------


class NotesError(Exception):
    """A refusal stated in the language of the tool, not of the machinery."""


# ---------------------------------------------------------------------------
# Streams — the three surfaces every command-line program stands on
# ---------------------------------------------------------------------------


@dataclass
class Streams:
    """Standard input, output and error, passed in rather than imported.

    A handler that writes to `streams.stdout` can be tested by handing it an
    io.StringIO. A handler that calls `print()` can only be tested by capturing
    the process. The difference is one parameter.
    """

    stdin: TextIO
    stdout: TextIO
    stderr: TextIO

    @classmethod
    def real(cls) -> "Streams":
        return cls(stdin=sys.stdin, stdout=sys.stdout, stderr=sys.stderr)

    def stdin_is_a_terminal(self) -> bool:
        """True when nothing is piped in — so the tool would hang waiting."""
        try:
            return self.stdin.isatty()
        except (AttributeError, ValueError):
            return False


# ---------------------------------------------------------------------------
# Custom argument types — conversion and validation in one place
# ---------------------------------------------------------------------------


def iso_date(text: str) -> date:
    """Convert YYYY-MM-DD to a date, or explain precisely why it cannot.

    Raising ArgumentTypeError is what makes argparse print a usage message on
    standard error and exit 2. Raising ValueError would give the user a
    traceback, which is a bug report addressed to the wrong person.
    """
    try:
        return datetime.strptime(text, "%Y-%m-%d").date()
    except ValueError:
        raise argparse.ArgumentTypeError(
            f"{text!r} is not a date in YYYY-MM-DD form (for example 2026-03-01)"
        ) from None


def positive_int(text: str) -> int:
    """Convert to an int that is at least 1."""
    try:
        value = int(text)
    except ValueError:
        raise argparse.ArgumentTypeError(f"{text!r} is not a whole number") from None
    if value < 1:
        raise argparse.ArgumentTypeError(f"{value} is not at least 1")
    return value


def tag_name(text: str) -> str:
    """A tag is lower case, non-empty, and has no spaces or commas."""
    cleaned = text.strip().lower()
    if not cleaned:
        raise argparse.ArgumentTypeError("a tag cannot be empty")
    if any(ch in cleaned for ch in " ,\t"):
        raise argparse.ArgumentTypeError(
            f"{text!r} is not a tag: tags contain no spaces or commas"
        )
    return cleaned


# ---------------------------------------------------------------------------
# The store — a small JSON file, kept behind two functions
# ---------------------------------------------------------------------------


def load_store(path: Path) -> dict[str, Any]:
    """Read the store, or return an empty one if the file is not there yet."""
    if not path.exists():
        return {"version": 1, "notes": []}
    try:
        data = json.loads(path.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        raise NotesError(f"{path} is not valid JSON ({exc.msg}, line {exc.lineno})") from None
    except OSError as exc:
        raise NotesError(f"cannot read {path}: {exc.strerror}") from None
    if not isinstance(data, dict) or not isinstance(data.get("notes"), list):
        raise NotesError(f"{path} does not look like a notes store")
    return data


def save_store(path: Path, data: dict[str, Any]) -> None:
    """Write the store back, formatted so a human can read the diff."""
    text = json.dumps(data, indent=2, ensure_ascii=False, sort_keys=True) + "\n"
    try:
        path.write_text(text, encoding="utf-8")
    except OSError as exc:
        raise NotesError(f"cannot write {path}: {exc.strerror}") from None


def next_id(notes: list[dict[str, Any]]) -> int:
    return max((int(note["id"]) for note in notes), default=0) + 1


def store_path(args: argparse.Namespace) -> Path:
    """Resolve the store path using the precedence users expect.

    flag beats environment beats built-in default. The flag's own default is
    None precisely so that "the user did not say" is distinguishable from
    "the user said the default".
    """
    if args.store is not None:
        return Path(args.store)
    from_env = os.environ.get(STORE_ENV_VAR)
    if from_env:
        return Path(from_env)
    return Path(DEFAULT_STORE)


# ---------------------------------------------------------------------------
# Rendering
# ---------------------------------------------------------------------------


def render_table(notes: list[dict[str, Any]], stream: TextIO) -> None:
    if not notes:
        return
    id_width = max(2, max(len(str(note["id"])) for note in notes))
    tag_width = max(4, max(len(",".join(note["tags"])) for note in notes))
    header = f"{'ID'.rjust(id_width)}  {'DATE':10}  {'TAGS'.ljust(tag_width)}  TEXT"
    stream.write(header + "\n")
    stream.write("-" * len(header) + "\n")
    for note in notes:
        tags = ",".join(note["tags"])
        stream.write(
            f"{str(note['id']).rjust(id_width)}  {note['date']:10}  "
            f"{tags.ljust(tag_width)}  {note['text']}\n"
        )


def render_json(notes: list[dict[str, Any]], stream: TextIO) -> None:
    json.dump(notes, stream, indent=2, ensure_ascii=False, sort_keys=True)
    stream.write("\n")


def render_csv(notes: list[dict[str, Any]], stream: TextIO) -> None:
    writer = csv.writer(stream, lineterminator="\n")
    writer.writerow(["id", "date", "tags", "text"])
    for note in notes:
        writer.writerow([note["id"], note["date"], ";".join(note["tags"]), note["text"]])


# ---------------------------------------------------------------------------
# Subcommand handlers — each returns an exit code, each takes its streams
# ---------------------------------------------------------------------------


def cmd_add(args: argparse.Namespace, streams: Streams) -> int:
    if args.text == "-":
        if streams.stdin_is_a_terminal():
            args.parser.error(
                "reading from standard input was requested with '-', "
                "but standard input is a terminal; pipe something in"
            )
        text = streams.stdin.read().strip()
        if not text:
            raise NotesError("standard input was empty, so there is nothing to add")
    else:
        text = args.text.strip()
        if not text:
            raise NotesError("a note cannot be empty")

    when = args.on if args.on is not None else date.today()
    path = store_path(args)
    data = load_store(path)
    note = {
        "id": next_id(data["notes"]),
        "date": when.isoformat(),
        "tags": sorted(set(args.tag or [])),
        "text": text,
    }
    data["notes"].append(note)

    if args.dry_run:
        if not args.quiet:
            streams.stdout.flush()
            streams.stderr.write(
                f"dry run: would add note {note['id']} to {path} (nothing was written)\n"
            )
        return 0

    save_store(path, data)
    if not args.quiet:
        streams.stdout.write(f"added note {note['id']}\n")
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"store: {path}\n")
    return 0


def select_notes(args: argparse.Namespace, notes: list[dict[str, Any]]) -> list[dict[str, Any]]:
    chosen = notes
    if args.tag:
        wanted = set(args.tag)
        chosen = [n for n in chosen if wanted.issubset(set(n["tags"]))]
    if getattr(args, "since", None) is not None:
        chosen = [n for n in chosen if n["date"] >= args.since.isoformat()]
    if getattr(args, "limit", None) is not None:
        chosen = chosen[-args.limit :]
    return chosen


def cmd_list(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    notes = select_notes(args, load_store(path)["notes"])
    if args.format == "json":
        render_json(notes, streams.stdout)
    else:
        render_table(notes, streams.stdout)
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"{len(notes)} note(s) from {path}\n")
    return 0


def cmd_search(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    notes = load_store(path)["notes"]
    needle = args.pattern.lower() if args.ignore_case else args.pattern
    matches = [
        note
        for note in notes
        if needle in (note["text"].lower() if args.ignore_case else note["text"])
    ]
    if args.format == "json":
        render_json(matches, streams.stdout)
    else:
        render_table(matches, streams.stdout)
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"{len(matches)} of {len(notes)} note(s) matched\n")
    return 0 if matches else 1


def cmd_export(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    notes = load_store(path)["notes"]
    if args.format == "csv":
        render_csv(notes, streams.stdout)
    else:
        render_json(notes, streams.stdout)
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"exported {len(notes)} note(s) as {args.format}\n")
    return 0


def cmd_remove(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    data = load_store(path)
    present = {int(note["id"]) for note in data["notes"]}
    missing = [str(i) for i in args.ids if i not in present]
    if missing:
        raise NotesError(f"no note with id {', '.join(missing)} in {path}")

    doomed = sorted(set(args.ids))
    if args.dry_run:
        for note_id in doomed:
            streams.stdout.write(f"would remove note {note_id}\n")
        streams.stdout.flush()
        streams.stderr.write(
            f"dry run: {len(doomed)} note(s) would be removed; {path} was not touched\n"
        )
        return 0

    data["notes"] = [n for n in data["notes"] if int(n["id"]) not in set(doomed)]
    save_store(path, data)
    if not args.quiet:
        for note_id in doomed:
            streams.stdout.write(f"removed note {note_id}\n")
    return 0


# ---------------------------------------------------------------------------
# The parser — a declaration of the tool's whole interface, in one place
# ---------------------------------------------------------------------------


def common_options() -> argparse.ArgumentParser:
    """Options shared by every subcommand.

    `add_help=False` matters: this parser is only ever used as a parent, and
    without it every subcommand would try to define -h twice and crash.
    """
    parent = argparse.ArgumentParser(add_help=False)

    storage = parent.add_argument_group(
        "storage options",
        f"where the notes live. Precedence: --store, then ${STORE_ENV_VAR}, "
        f"then ./{DEFAULT_STORE}",
    )
    storage.add_argument(
        "--store",
        metavar="PATH",
        default=None,
        help=f"path to the JSON store (default: ${STORE_ENV_VAR} or ./{DEFAULT_STORE})",
    )

    noise = parent.add_argument_group("output options")
    loudness = noise.add_mutually_exclusive_group()
    loudness.add_argument(
        "-v",
        "--verbose",
        action="store_true",
        help="explain what is happening, on standard error",
    )
    loudness.add_argument(
        "-q",
        "--quiet",
        action="store_true",
        help="print nothing on success",
    )
    return parent


def build_parser() -> argparse.ArgumentParser:
    """Build the parser. This function does no work and touches no file."""
    parent = common_options()

    parser = argparse.ArgumentParser(
        prog="notes",
        description="Keep short notes in a JSON file you can read with your own eyes.",
        epilog=(
            "Examples:\n"
            "  notes add 'ring the dentist' --tag health --on 2026-03-01\n"
            "  echo 'from a pipe' | notes add - --tag inbox\n"
            "  notes list --format json | python3 -m json.tool\n"
            "  notes remove 3 --dry-run\n"
            "\n"
            "Exit codes: 0 success, 1 refusal, 2 usage error."
        ),
        formatter_class=argparse.RawDescriptionHelpFormatter,
    )
    parser.add_argument(
        "--version",
        action="version",
        version=f"%(prog)s {__version__}",
        help="print the version and exit",
    )

    subcommands = parser.add_subparsers(
        title="subcommands",
        dest="command",
        metavar="<command>",
        required=True,
    )

    # --- add ---------------------------------------------------------------
    add = subcommands.add_parser(
        "add",
        parents=[parent],
        help="add one note",
        description="Add one note. Use '-' as the text to read it from standard input.",
    )
    add.add_argument("text", help="the note text, or '-' to read standard input")
    add.add_argument(
        "-t",
        "--tag",
        action="append",
        type=tag_name,
        metavar="TAG",
        help="attach a tag; repeat the option for more than one",
    )
    add.add_argument(
        "--on",
        type=iso_date,
        metavar="YYYY-MM-DD",
        help="the date to file the note under (default: today)",
    )
    add.add_argument(
        "--dry-run",
        action="store_true",
        help="say what would happen without writing anything",
    )
    add.set_defaults(func=cmd_add, parser=add)

    # --- list --------------------------------------------------------------
    listing = subcommands.add_parser(
        "list",
        parents=[parent],
        help="list notes",
        description="List notes, newest last.",
    )
    listing.add_argument(
        "--format",
        choices=["table", "json"],
        default="table",
        help="output format (default: table)",
    )
    listing.add_argument(
        "-t", "--tag", action="append", type=tag_name, metavar="TAG", help="only notes with this tag"
    )
    listing.add_argument("--since", type=iso_date, metavar="YYYY-MM-DD", help="only notes on or after this date")
    listing.add_argument("-n", "--limit", type=positive_int, metavar="N", help="show at most N notes")
    listing.set_defaults(func=cmd_list, parser=listing)

    # --- search ------------------------------------------------------------
    search = subcommands.add_parser(
        "search",
        parents=[parent],
        help="find notes containing text",
        description="Find notes whose text contains PATTERN. Exits 1 when nothing matched.",
    )
    search.add_argument("pattern", help="the text to look for")
    search.add_argument("-i", "--ignore-case", action="store_true", help="match regardless of case")
    search.add_argument(
        "--format", choices=["table", "json"], default="table", help="output format (default: table)"
    )
    search.set_defaults(func=cmd_search, parser=search)

    # --- export ------------------------------------------------------------
    export = subcommands.add_parser(
        "export",
        parents=[parent],
        help="write every note to standard output",
        description="Write every note to standard output so it can be piped or redirected.",
    )
    export.add_argument(
        "--format", choices=["json", "csv"], default="json", help="output format (default: json)"
    )
    export.set_defaults(func=cmd_export, parser=export)

    # --- remove ------------------------------------------------------------
    remove = subcommands.add_parser(
        "remove",
        parents=[parent],
        help="delete notes by id",
        description="Delete one or more notes. Destructive, so it has --dry-run.",
    )
    remove.add_argument("ids", nargs="+", type=positive_int, metavar="ID", help="note id to remove")
    remove.add_argument(
        "--dry-run",
        action="store_true",
        help="list what would be removed and leave the store untouched",
    )
    remove.set_defaults(func=cmd_remove, parser=remove)

    return parser


def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
    """Parse an EXPLICIT argument list.

    Defaulting to None (which argparse turns into sys.argv[1:]) keeps the real
    program convenient, while every test passes its own list and never has to
    monkey-patch a global.
    """
    return build_parser().parse_args(argv)


def main(argv: Sequence[str] | None = None, streams: Streams | None = None) -> int:
    streams = streams or Streams.real()
    args = parse_args(argv)
    handler: Callable[[argparse.Namespace, Streams], int] = args.func
    try:
        return handler(args, streams)
    except NotesError as exc:
        streams.stderr.write(f"notes: {exc}\n")
        return 1
    except BrokenPipeError:
        # `notes export | head -2` closes the pipe early. That is not an error.
        return 0
    except KeyboardInterrupt:
        streams.stderr.write("notes: interrupted\n")
        return 130


if __name__ == "__main__":
    sys.exit(main())
metadata.yml (1259 bytes)
lesson_id: D080
day: 80
kind: python-program
languages: [python]
setup_commands:
  - cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
  - python3 --version
  - python3 -m venv .venv
  - .venv/bin/pip install -r requirements/requirements.txt
run_commands:
  - python3 examples/by_hand.py
  - python3 examples/notes.py --help
  - python3 examples/notes.py add 'ring the dentist' --tag health --on 2026-03-01 --store notes.json
  - echo 'from a pipe' | python3 examples/notes.py add - --tag inbox --on 2026-03-03 --store notes.json
  - python3 examples/notes.py list --format json --store notes.json
  - python3 examples/notes.py remove 2 --dry-run --store notes.json
  - python3 starter/notes.py --help
test_commands:
  - bash tests/run_tests.sh
cleanup_commands:
  - rm -f notes.json
  - 'git checkout -- starter/  # optional: reset your work'
requires_network: true
requires_api_key: false
estimated_minutes: 30
last_executed: '2026-07-19'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1, bash 3.2.57 — bash tests/run_tests.sh -> 76 checks, 0 failure(s), exit 0 with the starter exercises unfinished (133 checks, 0 failure(s), exit 0 once all eight are completed); the in-process suite reports 42 passed'
requirements/README.md (2848 bytes)
# Dependencies — Day 080 lab

**One package, and it is only for the tests.**

The tool you build in this lab has **no third-party dependencies at all**.
`argparse` has been in the Python standard library since Python 3.2, released
in 2011, and so has everything else the tool touches. That is not an accident
of this lab's design — it is one of the strongest practical arguments for
argparse, and it is worth stating plainly rather than burying: a command-line
tool written with argparse can be handed to anyone with a Python interpreter
and it runs. No install step, no version pin, no network.

| Module | Used in | Why |
| --- | --- | --- |
| `argparse` | `notes.py` | the whole command-line interface: parser, subcommands, types, choices, groups, help |
| `json` | `notes.py` | the store format, and the `--format json` output |
| `csv` | `notes.py` | the `--format csv` export, quoted correctly by `csv.writer` |
| `os` | `notes.py` | reading `$NOTES_STORE` for the configuration-precedence rule |
| `sys` | `notes.py` | the three real streams and the process exit code |
| `pathlib` | `notes.py` | reading and writing the store file |
| `datetime` | `notes.py` | the `--on` date, converted by the custom `iso_date` type |
| `dataclasses` | `notes.py` | the small `Streams` record that carries stdin, stdout and stderr |

## What is in requirements.txt, and why

```text
pytest==9.1.1
```

`pytest` is needed only by `tests/test_parser.py`, the in-process half of the
suite — the part that calls `parse_args(["add", "hello"])` directly and
asserts on the resulting namespace without starting a process. You met pytest
on Day 71 and used it for the whole of Week 11.

pytest is free and open source (MIT licence). The exact version above is the
one the captured output in `expected-output/` came from; any recent pytest
will do, and the runner accepts an existing installation.

## Install

```bash
cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
```

`tests/run_tests.sh` looks for pytest in three places, in order: the `PYTEST`
environment variable, this lab's `.venv/bin/`, and then whatever is on your
`PATH`. If it finds none it prints the two commands above and exits non-zero —
it never skips silently, because a suite that quietly does nothing is worse
than a suite that fails.

If you already have pytest somewhere else:

```bash
PYTEST=/path/to/pytest bash tests/run_tests.sh
```

## No network, no account, no key

Nothing in this lab reaches the network at any point, including the install
step if you already have pytest. There is no account to create, no API key,
and nothing to pay for. The alternatives discussed in the lesson — `click`
and `typer` — are also free and open source, and neither is needed here.
requirements/requirements.txt (305 bytes)
# Day 080 lab — dependencies
#
# The TOOL you build needs nothing: argparse, json, csv, os, sys, pathlib and
# datetime are all Python standard library. The line below is for the TESTS.
#
# Install with:
#   python3 -m venv .venv
#   .venv/bin/pip install -r requirements/requirements.txt
pytest==9.1.1
starter/notes.py (21268 bytes)
#!/usr/bin/env python3
"""notes — YOUR working file for the Day 080 lab.

Eight numbered exercises build this into a tool you would actually install.
Everything that is NOT about the command-line interface — the JSON store, the
table and CSV renderers, the error type — is already written for you, because
today is about the interface, not about the storage.

Work top to bottom. After each exercise run:

    bash tests/run_tests.sh

The suite runs a structural pass while exercises remain, and switches to the
full behavioural battery — the same one the reference passes — the moment the
last NotImplementedError is gone.

The exercises, at a glance:

  1. iso_date        a custom type= that converts and validates a date
  2. positive_int    a second custom type=, for --limit and note ids
  3. common_options  an argument group and a mutually exclusive group
  4. build_parser    the root parser: prog, description, epilog, --version
  5. build_parser    the `add` subparser and its set_defaults dispatch
  6. build_parser    the list / search / export / remove subparsers
  7. cmd_add         reading the note from standard input when text is '-'
  8. cmd_remove      --dry-run: say what would happen, write nothing

Compare with examples/notes.py only AFTER you have tried. Reading the answer
first turns a two-hour lesson into a five-minute copy.
"""

from __future__ import annotations

import argparse
import csv
import json
import os
import sys
from dataclasses import dataclass
from datetime import date, datetime
from pathlib import Path
from typing import Any, Callable, Sequence, TextIO

__version__ = "1.0.0"

DEFAULT_STORE = "notes.json"
STORE_ENV_VAR = "NOTES_STORE"


# ===========================================================================
# PROVIDED — you do not need to change anything between here and EXERCISE 1
# ===========================================================================


class NotesError(Exception):
    """A refusal stated in the language of the tool, not of the machinery."""


@dataclass
class Streams:
    """Standard input, output and error, passed in rather than imported."""

    stdin: TextIO
    stdout: TextIO
    stderr: TextIO

    @classmethod
    def real(cls) -> "Streams":
        return cls(stdin=sys.stdin, stdout=sys.stdout, stderr=sys.stderr)

    def stdin_is_a_terminal(self) -> bool:
        """True when nothing is piped in — so the tool would hang waiting."""
        try:
            return self.stdin.isatty()
        except (AttributeError, ValueError):
            return False


def load_store(path: Path) -> dict[str, Any]:
    if not path.exists():
        return {"version": 1, "notes": []}
    try:
        data = json.loads(path.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        raise NotesError(f"{path} is not valid JSON ({exc.msg}, line {exc.lineno})") from None
    except OSError as exc:
        raise NotesError(f"cannot read {path}: {exc.strerror}") from None
    if not isinstance(data, dict) or not isinstance(data.get("notes"), list):
        raise NotesError(f"{path} does not look like a notes store")
    return data


def save_store(path: Path, data: dict[str, Any]) -> None:
    text = json.dumps(data, indent=2, ensure_ascii=False, sort_keys=True) + "\n"
    try:
        path.write_text(text, encoding="utf-8")
    except OSError as exc:
        raise NotesError(f"cannot write {path}: {exc.strerror}") from None


def next_id(notes: list[dict[str, Any]]) -> int:
    return max((int(note["id"]) for note in notes), default=0) + 1


def store_path(args: argparse.Namespace) -> Path:
    """Precedence: --store flag, then $NOTES_STORE, then ./notes.json."""
    if args.store is not None:
        return Path(args.store)
    from_env = os.environ.get(STORE_ENV_VAR)
    if from_env:
        return Path(from_env)
    return Path(DEFAULT_STORE)


def render_table(notes: list[dict[str, Any]], stream: TextIO) -> None:
    if not notes:
        return
    id_width = max(2, max(len(str(note["id"])) for note in notes))
    tag_width = max(4, max(len(",".join(note["tags"])) for note in notes))
    header = f"{'ID'.rjust(id_width)}  {'DATE':10}  {'TAGS'.ljust(tag_width)}  TEXT"
    stream.write(header + "\n")
    stream.write("-" * len(header) + "\n")
    for note in notes:
        tags = ",".join(note["tags"])
        stream.write(
            f"{str(note['id']).rjust(id_width)}  {note['date']:10}  "
            f"{tags.ljust(tag_width)}  {note['text']}\n"
        )


def render_json(notes: list[dict[str, Any]], stream: TextIO) -> None:
    json.dump(notes, stream, indent=2, ensure_ascii=False, sort_keys=True)
    stream.write("\n")


def render_csv(notes: list[dict[str, Any]], stream: TextIO) -> None:
    writer = csv.writer(stream, lineterminator="\n")
    writer.writerow(["id", "date", "tags", "text"])
    for note in notes:
        writer.writerow([note["id"], note["date"], ";".join(note["tags"]), note["text"]])


def tag_name(text: str) -> str:
    """A tag is lower case, non-empty, and has no spaces or commas."""
    cleaned = text.strip().lower()
    if not cleaned:
        raise argparse.ArgumentTypeError("a tag cannot be empty")
    if any(ch in cleaned for ch in " ,\t"):
        raise argparse.ArgumentTypeError(
            f"{text!r} is not a tag: tags contain no spaces or commas"
        )
    return cleaned


# ===========================================================================
# EXERCISE 1 — a custom type= that converts AND validates
# ===========================================================================
#
# argparse's `type=` is not only a converter; it is your validation hook. A
# function passed as `type=` receives the raw string, returns the converted
# value, and raises argparse.ArgumentTypeError when the string is unusable.
# argparse turns that exception into a usage message on STANDARD ERROR and
# exits 2. Raise a plain ValueError instead and the user gets a traceback.
#
# Write iso_date so that:
#   iso_date("2026-03-01")  ->  datetime.date(2026, 3, 1)
#   iso_date("2026-13-01")  ->  raises argparse.ArgumentTypeError
#   iso_date("tomorrow")    ->  raises argparse.ArgumentTypeError
#
# Hint: datetime.strptime(text, "%Y-%m-%d").date() raises ValueError on bad
# input. Catch it and re-raise as ArgumentTypeError with a message naming both
# the offending value and the form you wanted. `from None` suppresses the
# chained traceback, which the user does not need to see.
#
# Check it:
#   python3 -c "import notes; print(notes.iso_date('2026-03-01'))"


def iso_date(text: str) -> date:
    raise NotImplementedError("EXERCISE 1: convert YYYY-MM-DD or raise ArgumentTypeError")


# ===========================================================================
# EXERCISE 2 — a second custom type, for counts and ids
# ===========================================================================
#
# positive_int("3")   ->  3
# positive_int("0")   ->  raises argparse.ArgumentTypeError
# positive_int("-2")  ->  raises argparse.ArgumentTypeError
# positive_int("abc") ->  raises argparse.ArgumentTypeError
#
# Two different failures, two different messages: "is not a whole number" and
# "is not at least 1". A user who mistypes should not have to guess which.


def positive_int(text: str) -> int:
    raise NotImplementedError("EXERCISE 2: parse a whole number of at least 1")


# ===========================================================================
# PROVIDED — the handlers, except the two bodies you write in 7 and 8
# ===========================================================================


def cmd_add(args: argparse.Namespace, streams: Streams) -> int:
    if args.text == "-":
        # ===================================================================
        # EXERCISE 7 — read the note from standard input
        # ===================================================================
        #
        # `-` meaning "standard input" is a convention older than Python, and
        # following it is what lets your tool live inside a pipeline:
        #
        #     echo 'from a pipe' | python3 notes.py add - --tag inbox
        #
        # Two things must happen here.
        #
        # (a) TERMINAL DETECTION. If standard input is a terminal, nothing is
        #     piped in and reading would hang with no explanation. Call
        #     streams.stdin_is_a_terminal() and, when it is True, call
        #         args.parser.error("...")
        #     which prints usage on standard error and exits 2. Say what the
        #     user should have done, not just that they were wrong.
        #
        # (b) READ AND CHECK. Otherwise read streams.stdin.read(), strip it,
        #     and raise NotesError if it is empty — an empty pipe is a
        #     refusal, not a note.
        #
        # Assign the result to `text`.
        raise NotImplementedError("EXERCISE 7: read the note from standard input")
    else:
        text = args.text.strip()
        if not text:
            raise NotesError("a note cannot be empty")

    when = args.on if args.on is not None else date.today()
    path = store_path(args)
    data = load_store(path)
    note = {
        "id": next_id(data["notes"]),
        "date": when.isoformat(),
        "tags": sorted(set(args.tag or [])),
        "text": text,
    }
    data["notes"].append(note)

    if args.dry_run:
        if not args.quiet:
            streams.stdout.flush()
            streams.stderr.write(
                f"dry run: would add note {note['id']} to {path} (nothing was written)\n"
            )
        return 0

    save_store(path, data)
    if not args.quiet:
        streams.stdout.write(f"added note {note['id']}\n")
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"store: {path}\n")
    return 0


def select_notes(args: argparse.Namespace, notes: list[dict[str, Any]]) -> list[dict[str, Any]]:
    chosen = notes
    if args.tag:
        wanted = set(args.tag)
        chosen = [n for n in chosen if wanted.issubset(set(n["tags"]))]
    if getattr(args, "since", None) is not None:
        chosen = [n for n in chosen if n["date"] >= args.since.isoformat()]
    if getattr(args, "limit", None) is not None:
        chosen = chosen[-args.limit :]
    return chosen


def cmd_list(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    notes = select_notes(args, load_store(path)["notes"])
    if args.format == "json":
        render_json(notes, streams.stdout)
    else:
        render_table(notes, streams.stdout)
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"{len(notes)} note(s) from {path}\n")
    return 0


def cmd_search(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    notes = load_store(path)["notes"]
    needle = args.pattern.lower() if args.ignore_case else args.pattern
    matches = [
        note
        for note in notes
        if needle in (note["text"].lower() if args.ignore_case else note["text"])
    ]
    if args.format == "json":
        render_json(matches, streams.stdout)
    else:
        render_table(matches, streams.stdout)
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"{len(matches)} of {len(notes)} note(s) matched\n")
    # grep's convention: nothing found is exit 1, not an error message.
    return 0 if matches else 1


def cmd_export(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    notes = load_store(path)["notes"]
    if args.format == "csv":
        render_csv(notes, streams.stdout)
    else:
        render_json(notes, streams.stdout)
    if args.verbose:
        streams.stdout.flush()
        streams.stderr.write(f"exported {len(notes)} note(s) as {args.format}\n")
    return 0


def cmd_remove(args: argparse.Namespace, streams: Streams) -> int:
    path = store_path(args)
    data = load_store(path)
    present = {int(note["id"]) for note in data["notes"]}
    missing = [str(i) for i in args.ids if i not in present]
    if missing:
        raise NotesError(f"no note with id {', '.join(missing)} in {path}")

    doomed = sorted(set(args.ids))

    if args.dry_run:
        # =====================================================================
        # EXERCISE 8 — --dry-run on the one destructive command
        # =====================================================================
        #
        # A dry run is a promise: the tool tells you exactly what it would do
        # and changes nothing. Two rules make that promise keepable.
        #
        #   1. Do the READS and the CHECKS for real. Notice that the
        #      "no note with id ..." check above already ran — a dry run that
        #      skipped validation would happily promise an impossible deletion.
        #   2. Never reach the write. Return before save_store is called.
        #
        # Write, for each id in `doomed`, one line on STANDARD OUTPUT:
        #     would remove note 2
        # then one summary line on STANDARD ERROR:
        #     dry run: 1 note(s) would be removed; <path> was not touched
        # then return 0.
        #
        # The split is deliberate: the ids are the RESULT (a person could pipe
        # them into another command), the summary is a DIAGNOSTIC. Call
        # streams.stdout.flush() before writing to stderr so the two arrive in
        # the order you wrote them even when standard output is a pipe.
        #
        # The test suite checks the store is byte-for-byte identical
        # afterwards. That is the only check that can prove a dry run is real.
        raise NotImplementedError("EXERCISE 8: report what would happen, write nothing")

    data["notes"] = [n for n in data["notes"] if int(n["id"]) not in set(doomed)]
    save_store(path, data)
    if not args.quiet:
        for note_id in doomed:
            streams.stdout.write(f"removed note {note_id}\n")
    return 0


# ===========================================================================
# EXERCISE 3 — shared options: an argument group and a mutually exclusive pair
# ===========================================================================
#
# Every subcommand needs --store, --verbose and --quiet. Rather than repeating
# them five times, define them once on a PARENT parser and pass parents=[...]
# to each subparser. Note add_help=False: without it, every subcommand would
# try to define -h twice and argparse would raise at import time.
#
# Build, inside this function:
#
#   (a) an ARGUMENT GROUP called "storage options", created with
#       parent.add_argument_group(title, description). Groups do not change
#       parsing at all — they only change how --help is laid out, which is
#       reason enough, because your help output is the documentation most
#       users will ever read. Give it --store with metavar="PATH",
#       default=None, and a help string naming the precedence.
#
#       default=None is not laziness. It is how you tell "the user did not
#       say" apart from "the user said the default" — which is exactly what
#       store_path() above needs in order to consult $NOTES_STORE.
#
#   (b) a MUTUALLY EXCLUSIVE GROUP holding -v/--verbose and -q/--quiet, both
#       action="store_true". Created with
#       some_group.add_mutually_exclusive_group(). Passing both then fails
#       with exit code 2 and a message naming both options — argparse does
#       that for you, and a hand-rolled `if verbose and quiet` would not
#       produce a usage message.
#
# Return the parent parser.


def common_options() -> argparse.ArgumentParser:
    parent = argparse.ArgumentParser(add_help=False)
    raise NotImplementedError("EXERCISE 3: add the storage group and the verbose/quiet pair")


# ===========================================================================
# EXERCISES 4, 5 and 6 — the parser itself
# ===========================================================================


def build_parser() -> argparse.ArgumentParser:
    parent = common_options()

    # =======================================================================
    # EXERCISE 4 — the root parser
    # =======================================================================
    #
    # Create argparse.ArgumentParser with, at minimum:
    #   prog="notes"            so usage lines say `notes`, not `notes.py`
    #   description=...         one sentence, shown at the top of --help
    #   epilog=...              worked examples, shown at the bottom
    #   formatter_class=argparse.RawDescriptionHelpFormatter
    #                           so your epilog's line breaks survive
    #
    # Then add --version with action="version" and
    # version=f"%(prog)s {__version__}". `%(prog)s` is argparse's own
    # substitution, not an f-string field — it expands to the prog name.
    #
    # Users expect -h, --help and --version to exist. Two of the three you get
    # for free; the third is one line. Not having them is the first thing that
    # makes a tool feel homemade.
    raise NotImplementedError("EXERCISE 4: build the root parser and add --version")

    # =======================================================================
    # EXERCISE 5 — subcommands and the dispatch pattern
    # =======================================================================
    #
    # subcommands = parser.add_subparsers(
    #     title="subcommands", dest="command", metavar="<command>", required=True,
    # )
    #
    # required=True is what makes bare `notes` exit 2 with a usage message
    # instead of crashing with AttributeError when nothing set args.func.
    #
    # Then, for `add`:
    #
    #   add = subcommands.add_parser("add", parents=[parent],
    #                                help="add one note",
    #                                description="...")
    #   add.add_argument("text", help="the note text, or '-' to read stdin")
    #   add.add_argument("-t", "--tag", action="append", type=tag_name,
    #                    metavar="TAG", help="...")
    #   add.add_argument("--on", type=iso_date, metavar="YYYY-MM-DD", help="...")
    #   add.add_argument("--dry-run", action="store_true", help="...")
    #   add.set_defaults(func=cmd_add, parser=add)
    #
    # That last line is the whole dispatch pattern, and it is the single most
    # useful technique in this lesson. set_defaults stores arbitrary values on
    # the resulting namespace, so after parsing, `args.func` IS the right
    # handler. main() calls it without a single if-statement about which
    # subcommand ran. Storing `parser=add` as well gives the handler access to
    # its own parser so it can call parser.error() — which is how EXERCISE 7
    # reports a usage problem it could only detect at run time.
    #
    # Note also: `--dry-run` on the command line becomes `args.dry_run`.
    # argparse replaces dashes with underscores, because a dash is not legal
    # in a Python identifier.

    # =======================================================================
    # EXERCISE 6 — the remaining four subcommands
    # =======================================================================
    #
    #   list    --format {table,json} (default "table"), -t/--tag (append,
    #           type=tag_name), --since (type=iso_date),
    #           -n/--limit (type=positive_int)     -> cmd_list
    #   search  positional `pattern`; -i/--ignore-case (store_true);
    #           --format {table,json}              -> cmd_search
    #   export  --format {json,csv} (default "json")  -> cmd_export
    #   remove  positional `ids` with nargs="+" and type=positive_int;
    #           --dry-run (store_true)             -> cmd_remove
    #
    # Every one takes parents=[parent] and ends with set_defaults(func=...,
    # parser=...).
    #
    # `choices=` is worth pausing on: it validates, it prints the legal values
    # in the usage line, and a wrong value exits 2 with a message listing what
    # was allowed. Three jobs, one keyword.
    #
    # nargs="+" means one or more, collected into a list — so
    # `notes remove 2 5 9` deletes three notes with no extra parsing.
    #
    # Finally: return parser


def parse_args(argv: Sequence[str] | None = None) -> argparse.Namespace:
    """Parse an EXPLICIT argument list.

    Defaulting to None (which argparse turns into sys.argv[1:]) keeps the real
    program convenient, while every test passes its own list and never has to
    monkey-patch a global. This is Day 74's injection argument, applied to the
    command line.
    """
    return build_parser().parse_args(argv)


def main(argv: Sequence[str] | None = None, streams: Streams | None = None) -> int:
    streams = streams or Streams.real()
    args = parse_args(argv)
    handler: Callable[[argparse.Namespace, Streams], int] = args.func
    try:
        return handler(args, streams)
    except NotesError as exc:
        streams.stderr.write(f"notes: {exc}\n")
        return 1
    except BrokenPipeError:
        return 0
    except KeyboardInterrupt:
        streams.stderr.write("notes: interrupted\n")
        return 130


if __name__ == "__main__":
    sys.exit(main())
tests/conftest.py (1128 bytes)
"""Make the tool importable in-process.

Which copy is tested is chosen by the NOTES_DIR environment variable, so the
same suite can be pointed at examples/ (the reference) or at starter/ (your
work) without editing a line. `bash tests/run_tests.sh` sets it for you.

Importing the module rather than launching a process is the whole point of
the in-process tests: they exercise `parse_args(argv)` with an explicit list
and `main(argv, streams)` with io.StringIO streams, which is only possible
because neither function reaches for sys.argv or sys.stdout on its own.
"""

from __future__ import annotations

import os
import sys
from pathlib import Path

import pytest

LAB_DIR = Path(__file__).resolve().parent.parent
NOTES_DIR = LAB_DIR / os.environ.get("NOTES_DIR", "examples")

sys.path.insert(0, str(NOTES_DIR))


@pytest.fixture(scope="session")
def notes_module():
    """The module under test, imported once per session."""
    import notes

    return notes


@pytest.fixture()
def store(tmp_path) -> Path:
    """A fresh, empty store path inside pytest's own temporary directory."""
    return tmp_path / "notes.json"
tests/run_tests.sh (25036 bytes)
#!/usr/bin/env bash
# Tests for the Day 080 lab. Run from the lab directory:
#   bash tests/run_tests.sh
#
# A command-line interface is a contract with a SHELL, so most of this suite
# launches the real program as a subprocess and inspects what the shell can
# see: the exit code, standard output, and standard error — separately.
#
# Three checks are the ones worth reading, because they are the ones that
# cannot be faked:
#
#   * "a bad date writes to stderr, writes NOTHING to stdout, and exits 2"
#     captures the two streams into two different files. A program that
#     printed its error with print() would pass a combined-output check and
#     fail this one, which is exactly the point;
#   * "--dry-run leaves the store byte-identical" hashes the file before and
#     after. It is paired with a control that runs the same command WITHOUT
#     --dry-run and demands the hash changes, so the check cannot pass by the
#     program simply never writing anything;
#   * "parse_args works in-process with an explicit argv list" runs the pytest
#     suite in tests/, which never starts a process at all.
#
# No network, no clock beyond an explicit --on, deterministic. Exits 0 only if
# every check passes.
set -u

export PYTHONDONTWRITEBYTECODE=1

lab_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
failures=0
checks=0

check() {
  local label="$1" ok="$2"
  checks=$((checks + 1))
  if [ "${ok}" = "yes" ]; then
    echo "  ok: ${label}"
  else
    echo "  FAIL: ${label}"
    failures=$((failures + 1))
  fi
}

# Resolve tools: an explicit override, then this lab's .venv, then PATH.
# Fails loudly with install instructions rather than silently skipping.
resolve_tool() {
  local tool="$1" override="$2"
  if [ -n "${override}" ] && [ -x "${override}" ]; then echo "${override}"; return 0; fi
  if [ -x "${lab_dir}/.venv/bin/${tool}" ]; then echo "${lab_dir}/.venv/bin/${tool}"; return 0; fi
  if command -v "${tool}" >/dev/null 2>&1; then command -v "${tool}"; return 0; fi
  return 1
}

python_bin="$(resolve_tool python3 "${PYTHON:-}")" || {
  echo "FAIL: python3 not found on PATH." >&2
  exit 1
}

pytest_bin="$(resolve_tool pytest "${PYTEST:-}")" || {
  echo "FAIL: pytest not found." >&2
  echo "  Install it with:" >&2
  echo "    python3 -m venv .venv" >&2
  echo "    .venv/bin/pip install -r requirements/requirements.txt" >&2
  echo "  Or point this suite at an existing pytest: PYTEST=/path/to/pytest bash tests/run_tests.sh" >&2
  exit 1
}

work="$(mktemp -d "${TMPDIR:-/tmp}/notes-cli.XXXXXX")"
cleanup() { rm -rf "${work}"; }
trap cleanup EXIT

hash_of() { "${python_bin}" -c "
import hashlib, sys, pathlib
print(hashlib.sha256(pathlib.Path(sys.argv[1]).read_bytes()).hexdigest())
" "$1"; }

echo "Day 080 — A Tool You Would Actually Install"
echo

# ==========================================================================
# The behavioural battery. Everything below runs against whichever copy of
# notes.py it is handed, so the reference and your finished starter are held
# to exactly the same standard.
# ==========================================================================

run_battery() {
  local label="$1" program="$2"
  local dir="${work}/${label}"
  local store="${dir}/store.json"
  local out="${dir}/out.txt" err="${dir}/err.txt"
  mkdir -p "${dir}"

  local notes="${python_bin} ${program}"

  # ------------------------------------------------------------------------
  # 1. Help output — the documentation most users will ever read
  # ------------------------------------------------------------------------

  ${notes} --help >"${out}" 2>"${err}"
  local rc=$?
  [ "${rc}" -eq 0 ] &&
    check "${label}: --help exits 0" "yes" ||
    check "${label}: --help exits 0 (got ${rc})" "no"

  [ -s "${out}" ] &&
    check "${label}: --help writes to standard output" "yes" ||
    check "${label}: --help writes to standard output" "no"

  [ ! -s "${err}" ] &&
    check "${label}: --help writes nothing to standard error" "yes" ||
    check "${label}: --help writes nothing to standard error" "no"

  local subcommand
  for subcommand in add list search export remove; do
    grep -q "^ *${subcommand} " "${out}" &&
      check "${label}: --help names the '${subcommand}' subcommand" "yes" ||
      check "${label}: --help names the '${subcommand}' subcommand" "no"
  done

  grep -q "usage: notes" "${out}" &&
    check "${label}: usage line says 'notes', not the file name (prog=)" "yes" ||
    check "${label}: usage line says 'notes', not the file name (prog=)" "no"

  grep -q "Exit codes" "${out}" &&
    check "${label}: the epilog survives with its line breaks intact" "yes" ||
    check "${label}: the epilog survives with its line breaks intact" "no"

  ${notes} add --help >"${out}" 2>&1
  grep -q "YYYY-MM-DD" "${out}" &&
    check "${label}: 'add --help' shows the --on metavar" "yes" ||
    check "${label}: 'add --help' shows the --on metavar" "no"
  grep -q "storage options" "${out}" &&
    check "${label}: 'add --help' groups the storage options" "yes" ||
    check "${label}: 'add --help' groups the storage options" "no"

  ${notes} --version >"${out}" 2>&1
  rc=$?
  { [ "${rc}" -eq 0 ] && grep -q "^notes 1\.0\.0$" "${out}"; } &&
    check "${label}: --version prints 'notes 1.0.0' and exits 0" "yes" ||
    check "${label}: --version prints 'notes 1.0.0' and exits 0 (got ${rc})" "no"

  # ------------------------------------------------------------------------
  # 2. Usage errors — argparse's exit code 2, and nothing on stdout
  # ------------------------------------------------------------------------

  ${notes} >"${out}" 2>"${err}"
  rc=$?
  [ "${rc}" -eq 2 ] &&
    check "${label}: no subcommand at all exits 2" "yes" ||
    check "${label}: no subcommand at all exits 2 (got ${rc})" "no"

  ${notes} frobnicate >"${out}" 2>"${err}"
  rc=$?
  [ "${rc}" -ne 0 ] &&
    check "${label}: an unknown subcommand exits non-zero (${rc})" "yes" ||
    check "${label}: an unknown subcommand exits non-zero" "no"
  grep -q "invalid choice" "${err}" &&
    check "${label}: an unknown subcommand is explained on standard error" "yes" ||
    check "${label}: an unknown subcommand is explained on standard error" "no"

  ${notes} add "x" --tagg typo --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 2 ] && [ ! -s "${out}" ]; } &&
    check "${label}: a mistyped option is refused, not ignored" "yes" ||
    check "${label}: a mistyped option is refused, not ignored (got ${rc})" "no"

  ${notes} list -v -q --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 2 ] && grep -q "not allowed with argument" "${err}"; } &&
    check "${label}: -v and -q together exit 2, naming both options" "yes" ||
    check "${label}: -v and -q together exit 2, naming both options" "no"

  ${notes} list --format yaml --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 2 ] && grep -q "invalid choice" "${err}"; } &&
    check "${label}: a value outside choices= exits 2" "yes" ||
    check "${label}: a value outside choices= exits 2" "no"

  ${notes} list -n 0 --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 2 ] && grep -q "at least 1" "${err}"; } &&
    check "${label}: the custom positive_int type rejects 0" "yes" ||
    check "${label}: the custom positive_int type rejects 0" "no"

  # ---- THE STREAM-SEPARATION CHECK ---------------------------------------
  # Two separate files, two separate assertions. A program that printed its
  # error message on standard output would pass a 2>&1 check and fail here.
  ${notes} add "x" --on 2026-13-01 --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  [ "${rc}" -ne 0 ] &&
    check "${label}: a bad date exits non-zero (${rc})" "yes" ||
    check "${label}: a bad date exits non-zero" "no"
  [ "${rc}" -eq 2 ] &&
    check "${label}: a bad date exits exactly 2 — argparse's usage code" "yes" ||
    check "${label}: a bad date exits exactly 2 (got ${rc})" "no"
  [ ! -s "${out}" ] &&
    check "${label}: a bad date writes NOTHING to standard output" "yes" ||
    check "${label}: a bad date writes NOTHING to standard output" "no"
  [ -s "${err}" ] &&
    check "${label}: a bad date writes a message to standard error" "yes" ||
    check "${label}: a bad date writes a message to standard error" "no"
  grep -q "2026-13-01" "${err}" &&
    check "${label}: the message quotes the value that was rejected" "yes" ||
    check "${label}: the message quotes the value that was rejected" "no"
  grep -q "YYYY-MM-DD" "${err}" &&
    check "${label}: the message says what a date should look like" "yes" ||
    check "${label}: the message says what a date should look like" "no"
  [ ! -e "${store}" ] &&
    check "${label}: a refused command created no store file" "yes" ||
    check "${label}: a refused command created no store file" "no"

  ${notes} add "x" --tag "two words" --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 2 ] && grep -q "no spaces or commas" "${err}"; } &&
    check "${label}: the custom tag type rejects a tag with a space" "yes" ||
    check "${label}: the custom tag type rejects a tag with a space" "no"

  # ------------------------------------------------------------------------
  # 3. The happy path
  # ------------------------------------------------------------------------

  ${notes} add "ring the dentist" --tag health --on 2026-03-01 --store "${store}" \
    >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 0 ] && grep -q "^added note 1$" "${out}" && [ ! -s "${err}" ]; } &&
    check "${label}: a successful add exits 0, reports on stdout, is silent on stderr" "yes" ||
    check "${label}: a successful add exits 0, reports on stdout, is silent on stderr (got ${rc})" "no"

  ${notes} add "argparse turns a script into a tool" -t python -t writing \
    --on 2026-03-02 --store "${store}" >/dev/null 2>&1
  check "${label}: --tag repeats (action='append')" \
    "$(${notes} list --format json --store "${store}" 2>/dev/null |
       grep -q '"python"' && echo yes || echo no)"

  # ---- stdin, so the tool composes in a pipeline --------------------------
  echo "from a pipe" | ${notes} add - --tag inbox --on 2026-03-03 --store "${store}" \
    >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 0 ] && grep -q "^added note 3$" "${out}"; } &&
    check "${label}: 'add -' reads the note from a pipe" "yes" ||
    check "${label}: 'add -' reads the note from a pipe (got ${rc})" "no"
  ${notes} search "from a pipe" --store "${store}" >"${out}" 2>&1
  grep -q "from a pipe" "${out}" &&
    check "${label}: the piped text was really stored" "yes" ||
    check "${label}: the piped text was really stored" "no"

  # ---- a double dash ends option parsing ---------------------------------
  ${notes} add --on 2026-03-04 --store "${store}" -- "--not-a-flag" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 0 ] && grep -q "^added note 4$" "${out}"; } &&
    check "${label}: '--' lets a note start with two dashes" "yes" ||
    check "${label}: '--' lets a note start with two dashes (got ${rc})" "no"

  # ---- output formats -----------------------------------------------------
  ${notes} list --format json --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 0 ] && "${python_bin}" -c "
import json, sys
notes = json.load(open(sys.argv[1]))
assert isinstance(notes, list) and len(notes) == 4, notes
assert notes[0]['text'] == 'ring the dentist', notes[0]
" "${out}"; } &&
    check "${label}: 'list --format json' emits parseable JSON on stdout" "yes" ||
    check "${label}: 'list --format json' emits parseable JSON on stdout" "no"

  ${notes} list --store "${store}" >"${out}" 2>&1
  grep -q "^ID  DATE" "${out}" &&
    check "${label}: 'list' defaults to the table format" "yes" ||
    check "${label}: 'list' defaults to the table format" "no"

  ${notes} list -n 1 --store "${store}" >"${out}" 2>&1
  [ "$(grep -c '2026-03' "${out}")" -eq 1 ] &&
    check "${label}: '-n 1' limits the listing to one note" "yes" ||
    check "${label}: '-n 1' limits the listing to one note" "no"

  ${notes} export --format csv --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 0 ] && head -1 "${out}" | grep -q "^id,date,tags,text$" &&
    [ "$(wc -l <"${out}")" -eq 5 ]; } &&
    check "${label}: 'export --format csv' writes a header and four rows" "yes" ||
    check "${label}: 'export --format csv' writes a header and four rows" "no"

  # ---- diagnostics on stderr keep the RESULT on stdout clean --------------
  ${notes} list --format json -v --store "${store}" >"${out}" 2>"${err}"
  { "${python_bin}" -c "import json,sys; json.load(open(sys.argv[1]))" "${out}" &&
    grep -q "4 note(s) from" "${err}"; } &&
    check "${label}: -v chatter goes to stderr, leaving stdout valid JSON" "yes" ||
    check "${label}: -v chatter goes to stderr, leaving stdout valid JSON" "no"

  ${notes} add "quiet one" --on 2026-03-05 -q --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 0 ] && [ ! -s "${out}" ] && [ ! -s "${err}" ]; } &&
    check "${label}: -q succeeds in silence on both streams" "yes" ||
    check "${label}: -q succeeds in silence on both streams" "no"

  # ---- search follows grep's exit-code convention -------------------------
  ${notes} search dentist --store "${store}" >/dev/null 2>&1
  rc=$?
  [ "${rc}" -eq 0 ] &&
    check "${label}: search exits 0 when something matched" "yes" ||
    check "${label}: search exits 0 when something matched (got ${rc})" "no"
  ${notes} search kangaroo --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 1 ] && [ ! -s "${out}" ]; } &&
    check "${label}: search exits 1 when nothing matched, printing nothing" "yes" ||
    check "${label}: search exits 1 when nothing matched (got ${rc})" "no"
  ${notes} search DENTIST -i --store "${store}" >/dev/null 2>&1
  [ $? -eq 0 ] &&
    check "${label}: '-i' matches regardless of case" "yes" ||
    check "${label}: '-i' matches regardless of case" "no"

  # ------------------------------------------------------------------------
  # 4. --dry-run — the promise, and the control that proves it means anything
  # ------------------------------------------------------------------------

  local before after
  before="$(hash_of "${store}")"
  ${notes} remove 2 --dry-run --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  after="$(hash_of "${store}")"

  [ "${rc}" -eq 0 ] &&
    check "${label}: 'remove --dry-run' exits 0" "yes" ||
    check "${label}: 'remove --dry-run' exits 0 (got ${rc})" "no"
  [ "${before}" = "${after}" ] &&
    check "${label}: 'remove --dry-run' leaves the store BYTE-IDENTICAL" "yes" ||
    check "${label}: 'remove --dry-run' leaves the store BYTE-IDENTICAL" "no"
  grep -q "^would remove note 2$" "${out}" &&
    check "${label}: the dry run puts the ids on stdout, where they can be piped" "yes" ||
    check "${label}: the dry run puts the ids on stdout, where they can be piped" "no"
  grep -q "was not touched" "${err}" &&
    check "${label}: the dry run puts its summary on stderr, where it is a diagnostic" "yes" ||
    check "${label}: the dry run puts its summary on stderr, where it is a diagnostic" "no"

  # A dry run still VALIDATES: an impossible deletion is refused, not promised.
  ${notes} remove 999 --dry-run --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 1 ] && [ ! -s "${out}" ] && grep -q "no note with id 999" "${err}"; } &&
    check "${label}: a dry run still refuses a note that does not exist" "yes" ||
    check "${label}: a dry run still refuses a note that does not exist (got ${rc})" "no"

  # THE CONTROL: without --dry-run the same command must change the file.
  # Without this, "the bytes did not change" could be satisfied by a program
  # that never writes at all.
  before="$(hash_of "${store}")"
  ${notes} remove 2 --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  after="$(hash_of "${store}")"
  { [ "${rc}" -eq 0 ] && [ "${before}" != "${after}" ] &&
    grep -q "^removed note 2$" "${out}"; } &&
    check "${label}: control — a real remove DOES change the store" "yes" ||
    check "${label}: control — a real remove DOES change the store" "no"

  ${notes} remove 2 --store "${store}" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 1 ] && [ ! -s "${out}" ] && grep -q "no note with id 2" "${err}"; } &&
    check "${label}: removing it twice is a refusal on stderr with exit 1" "yes" ||
    check "${label}: removing it twice is a refusal on stderr with exit 1 (got ${rc})" "no"

  ${notes} remove 1 3 --store "${store}" >"${out}" 2>&1
  [ "$(grep -c '^removed note' "${out}")" -eq 2 ] &&
    check "${label}: nargs='+' removes several notes in one command" "yes" ||
    check "${label}: nargs='+' removes several notes in one command" "no"

  # ------------------------------------------------------------------------
  # 5. Configuration precedence, and composing in a pipeline
  # ------------------------------------------------------------------------

  local env_store="${dir}/from-env.json"
  ( cd "${dir}" && NOTES_STORE="${env_store}" ${notes} add "via the environment" \
      --on 2026-03-06 >/dev/null 2>&1 )
  [ -f "${env_store}" ] &&
    check "${label}: \$NOTES_STORE is used when --store is absent" "yes" ||
    check "${label}: \$NOTES_STORE is used when --store is absent" "no"

  local flag_store="${dir}/from-flag.json"
  ( cd "${dir}" && NOTES_STORE="${env_store}" ${notes} add "via the flag" \
      --on 2026-03-06 --store "${flag_store}" >/dev/null 2>&1 )
  { [ -f "${flag_store}" ] &&
    [ "$(${notes} list --format json --store "${env_store}" 2>/dev/null |
         grep -c 'via the flag')" -eq 0 ]; } &&
    check "${label}: --store beats \$NOTES_STORE (flag wins)" "yes" ||
    check "${label}: --store beats \$NOTES_STORE (flag wins)" "no"

  # Note the store for this battery is store.json, so ./notes.json appearing
  # here can only have come from the built-in default.
  ( cd "${dir}" && ${notes} add "the built-in default" --on 2026-03-06 >/dev/null 2>&1 )
  [ -f "${dir}/notes.json" ] &&
    check "${label}: with neither, the built-in ./notes.json default applies" "yes" ||
    check "${label}: with neither, the built-in ./notes.json default applies" "no"

  # A tool that writes its result to stdout can be piped into another tool.
  local piped
  piped="$(${notes} export --format json --store "${store}" 2>/dev/null |
           "${python_bin}" -c "import json,sys; print(len(json.load(sys.stdin)))")"
  [ "${piped}" = "2" ] &&
    check "${label}: 'export | python3' composes — stdout is machine-readable" "yes" ||
    check "${label}: 'export | python3' composes (got '${piped}')" "no"

  # Closing the pipe early must not produce a BrokenPipeError traceback.
  ${notes} export --format csv --store "${store}" 2>"${err}" | head -1 >/dev/null
  [ ! -s "${err}" ] &&
    check "${label}: closing the pipe early is not an error" "yes" ||
    check "${label}: closing the pipe early is not an error" "no"

  # ------------------------------------------------------------------------
  # 6. Refusals that are not usage errors exit 1, not 2
  # ------------------------------------------------------------------------

  echo "{ this is not json" >"${dir}/broken.json"
  ${notes} list --store "${dir}/broken.json" >"${out}" 2>"${err}"
  rc=$?
  { [ "${rc}" -eq 1 ] && [ ! -s "${out}" ] && grep -q "not valid JSON" "${err}"; } &&
    check "${label}: a corrupt store is exit 1 on stderr, not a traceback" "yes" ||
    check "${label}: a corrupt store is exit 1 on stderr, not a traceback (got ${rc})" "no"
  grep -q "Traceback" "${err}" &&
    check "${label}: no Python traceback ever reaches the user" "no" ||
    check "${label}: no Python traceback ever reaches the user" "yes"
}

# --------------------------------------------------------------------------
echo "1. The reference tool (examples/notes.py)"
# --------------------------------------------------------------------------
run_battery "reference" "${lab_dir}/examples/notes.py"

# --------------------------------------------------------------------------
echo
echo "2. The hand-rolled parser, and why it is not enough"
# --------------------------------------------------------------------------

by_hand_out="${work}/by_hand.txt"
"${python_bin}" "${lab_dir}/examples/by_hand.py" >"${by_hand_out}" 2>&1
by_hand_rc=$?
[ "${by_hand_rc}" -eq 0 ] &&
  check "examples/by_hand.py runs and exits 0" "yes" ||
  check "examples/by_hand.py runs and exits 0 (got ${by_hand_rc})" "no"
grep -q "8 ordinary command lines, 4 of them handled wrongly." "${by_hand_out}" &&
  check "the hand-rolled parser gets 4 of 8 ordinary command lines wrong" "yes" ||
  check "the hand-rolled parser gets 4 of 8 ordinary command lines wrong" "no"
grep -q "the tag is silently lost" "${by_hand_out}" &&
  check "it loses the value of --tag=shopping without saying so" "yes" ||
  check "it loses the value of --tag=shopping without saying so" "no"
grep -q "IndexError" "${by_hand_out}" &&
  check "a missing option value gives an IndexError, not a usage message" "yes" ||
  check "a missing option value gives an IndexError, not a usage message" "no"

# argparse handles every one of those correctly. Two spot-checks:
ref="${lab_dir}/examples/notes.py"
spot_store="${work}/spot.json"
"${python_bin}" "${ref}" add "buy milk" --tag=shopping --on 2026-03-01 \
  --store "${spot_store}" >/dev/null 2>&1
"${python_bin}" "${ref}" list --format json --store "${spot_store}" 2>/dev/null |
  grep -q '"shopping"' &&
  check "argparse handles --tag=shopping, which the hand-rolled parser dropped" "yes" ||
  check "argparse handles --tag=shopping, which the hand-rolled parser dropped" "no"
"${python_bin}" "${ref}" add "x" --tag --store "${spot_store}" >/dev/null 2>"${work}/e.txt"
spot_rc=$?
{ [ "${spot_rc}" -eq 2 ] && grep -q "expected one argument" "${work}/e.txt"; } &&
  check "argparse turns a missing option value into a usage message and exit 2" "yes" ||
  check "argparse turns a missing option value into a usage message and exit 2" "no"

# --------------------------------------------------------------------------
echo
echo "3. In-process: parse_args with an explicit argv list"
# --------------------------------------------------------------------------

pytest_out="$(cd "${lab_dir}" && NOTES_DIR=examples "${pytest_bin}" tests -q 2>&1)"
pytest_rc=$?
[ "${pytest_rc}" -eq 0 ] &&
  check "the in-process pytest suite passes against examples/" "yes" ||
  check "the in-process pytest suite passes against examples/ (exit ${pytest_rc})" "no"
if [ "${pytest_rc}" -ne 0 ]; then printf '%s\n' "${pytest_out}" | tail -25; fi
printf '%s' "${pytest_out}" | grep -qE '[0-9]+ passed' &&
  check "it reports passing tests ( $(printf '%s' "${pytest_out}" | tail -1) )" "yes" ||
  check "it reports passing tests" "no"

# The in-process suite must be testing something. Break the parser's prog name
# in a COPY and demand that the subprocess battery would notice.
sandbox="${work}/sandbox"
mkdir -p "${sandbox}"
cp "${ref}" "${sandbox}/notes.py"
sed -i.bak 's/prog="notes"/prog="notez"/' "${sandbox}/notes.py"
rm -f "${sandbox}/notes.py.bak"
"${python_bin}" "${sandbox}/notes.py" --help 2>&1 | grep -q "usage: notes " &&
  check "a broken prog= would be caught (it was not — the check is vacuous)" "no" ||
  check "a one-line break to prog= changes the help output, so the check bites" "yes"

# --------------------------------------------------------------------------
echo
echo "4. The starter"
# --------------------------------------------------------------------------

starter="${lab_dir}/starter/notes.py"

"${python_bin}" -c "
import ast, sys
source = open(sys.argv[1], encoding='utf-8').read()
tree = ast.parse(source)
names = {n.name for n in ast.walk(tree) if isinstance(n, ast.FunctionDef)}
required = {
    'iso_date', 'positive_int', 'tag_name', 'common_options', 'build_parser',
    'parse_args', 'main', 'cmd_add', 'cmd_list', 'cmd_search', 'cmd_export',
    'cmd_remove',
}
missing = required - names
assert not missing, f'starter is missing: {sorted(missing)}'
" "${starter}"
[ $? -eq 0 ] &&
  check "starter/notes.py is valid Python and defines every required function" "yes" ||
  check "starter/notes.py is valid Python and defines every required function" "no"

for n in 1 2 3 4 5 6 7 8; do
  grep -q "EXERCISE ${n}" "${starter}" &&
    check "starter/notes.py carries EXERCISE ${n}" "yes" ||
    check "starter/notes.py carries EXERCISE ${n}" "no"
done

grep -q "argparse" "${starter}" &&
  check "starter/notes.py imports argparse" "yes" ||
  check "starter/notes.py imports argparse" "no"

remaining="$(grep -c 'raise NotImplementedError' "${starter}" || true)"
if [ "${remaining}" -gt 0 ]; then
  echo "  ..  ${remaining} exercise(s) still unfinished — the behavioural battery"
  echo "      will run against starter/notes.py once they are done."
else
  echo
  echo "5. Your finished starter, held to the same standard"
  run_battery "starter" "${starter}"
fi

echo
echo "${checks} checks, ${failures} failure(s)."
[ "${failures}" -eq 0 ]
tests/test_parser.py (10515 bytes)
"""In-process tests: the parser and the handlers, with no subprocess at all.

Read this file alongside `run_tests.sh`. The two prove different things:

  * run_tests.sh launches the real program and checks exit codes and the two
    output streams — the contract a SHELL sees;
  * this file calls parse_args(argv) and main(argv, streams) directly — the
    contract PYTHON sees.

You need both. A subprocess test is the only honest way to check an exit code
or stream separation. An in-process test is a hundred times faster, gives a
real traceback when it fails, and can assert on the parsed Namespace, which a
subprocess can never see.

That both are possible at all comes down to two design decisions in notes.py:
`parse_args` takes an explicit argument list instead of reading sys.argv, and
`main` takes a Streams object instead of importing sys.stdout. Day 74 called
that injecting the boundary; this is the same idea, one layer out.
"""

from __future__ import annotations

import argparse
import io
import json

import pytest


# ---------------------------------------------------------------------------
# parse_args: the parser as a pure function from a list of strings
# ---------------------------------------------------------------------------


def test_parse_args_takes_an_explicit_list(notes_module):
    args = notes_module.parse_args(["add", "hello"])
    assert args.command == "add"
    assert args.text == "hello"


def test_the_subcommand_binds_its_handler(notes_module):
    """set_defaults(func=...) is the dispatch: no if-statement anywhere."""
    for name, handler in [
        ("add", "cmd_add"),
        ("list", "cmd_list"),
        ("search", "cmd_search"),
        ("export", "cmd_export"),
        ("remove", "cmd_remove"),
    ]:
        argv = {"add": ["add", "x"], "search": ["search", "x"], "remove": ["remove", "1"]}.get(
            name, [name]
        )
        args = notes_module.parse_args(argv)
        assert args.func is getattr(notes_module, handler)


def test_dashes_become_underscores(notes_module):
    args = notes_module.parse_args(["remove", "3", "--dry-run"])
    assert args.dry_run is True
    assert args.ids == [3]


def test_nargs_plus_collects_a_list(notes_module):
    args = notes_module.parse_args(["remove", "2", "5", "9"])
    assert args.ids == [2, 5, 9]


def test_action_append_collects_repeats(notes_module):
    args = notes_module.parse_args(["add", "x", "-t", "one", "--tag", "two"])
    assert args.tag == ["one", "two"]


def test_defaults_apply_when_the_option_is_absent(notes_module):
    args = notes_module.parse_args(["list"])
    assert args.format == "table"
    assert args.store is None  # None, not "notes.json": see store_path()
    assert args.verbose is False and args.quiet is False


def test_type_conversion_happens_during_parsing(notes_module):
    from datetime import date

    args = notes_module.parse_args(["add", "x", "--on", "2026-03-01"])
    assert args.on == date(2026, 3, 1)
    assert isinstance(args.on, date)


def test_a_double_dash_ends_option_parsing(notes_module):
    args = notes_module.parse_args(["add", "--", "--not-a-flag"])
    assert args.text == "--not-a-flag"


@pytest.mark.parametrize(
    "argv",
    [
        [],  # no subcommand at all
        ["frobnicate"],  # unknown subcommand
        ["add"],  # missing required positional
        ["add", "x", "--on", "2026-13-01"],  # custom type rejects the value
        ["add", "x", "--tag", "two words"],  # custom type rejects the value
        ["list", "--format", "yaml"],  # not in choices
        ["list", "-n", "0"],  # positive_int rejects it
        ["list", "-v", "-q"],  # mutually exclusive
        ["remove"],  # nargs="+" needs at least one
        ["add", "x", "--tagg", "typo"],  # unknown option is refused, not ignored
    ],
)
def test_usage_errors_raise_systemexit_with_code_2(notes_module, argv):
    """argparse's contract: a usage error is SystemExit(2), never a traceback."""
    with pytest.raises(SystemExit) as caught:
        notes_module.parse_args(argv)
    assert caught.value.code == 2


def test_help_and_version_exit_zero(notes_module):
    for argv in (["--help"], ["--version"], ["add", "--help"]):
        with pytest.raises(SystemExit) as caught:
            notes_module.parse_args(argv)
        assert caught.value.code == 0


# ---------------------------------------------------------------------------
# The custom types, tested on their own — they are just functions
# ---------------------------------------------------------------------------


def test_iso_date_converts(notes_module):
    from datetime import date

    assert notes_module.iso_date("2026-03-01") == date(2026, 3, 1)


@pytest.mark.parametrize("bad", ["2026-13-01", "tomorrow", "01-03-2026", "", "2026-02-30"])
def test_iso_date_raises_argument_type_error(notes_module, bad):
    with pytest.raises(argparse.ArgumentTypeError):
        notes_module.iso_date(bad)


@pytest.mark.parametrize("bad", ["0", "-2", "abc", "1.5"])
def test_positive_int_raises_argument_type_error(notes_module, bad):
    with pytest.raises(argparse.ArgumentTypeError):
        notes_module.positive_int(bad)


def test_the_error_message_names_the_offending_value(notes_module):
    with pytest.raises(argparse.ArgumentTypeError) as caught:
        notes_module.iso_date("2026-13-01")
    assert "2026-13-01" in str(caught.value)


# ---------------------------------------------------------------------------
# main() driven with injected streams — no process, no real terminal
# ---------------------------------------------------------------------------


def run(notes_module, argv, stdin_text=""):
    """Call main() with string streams and return (code, stdout, stderr)."""
    streams = notes_module.Streams(
        stdin=io.StringIO(stdin_text), stdout=io.StringIO(), stderr=io.StringIO()
    )
    code = notes_module.main(argv, streams)
    return code, streams.stdout.getvalue(), streams.stderr.getvalue()


def test_add_then_list_round_trips(notes_module, store):
    code, out, err = run(notes_module, ["add", "buy milk", "--on", "2026-03-01", "--store", str(store)])
    assert (code, out, err) == (0, "added note 1\n", "")

    code, out, err = run(notes_module, ["list", "--format", "json", "--store", str(store)])
    assert code == 0
    assert json.loads(out) == [{"date": "2026-03-01", "id": 1, "tags": [], "text": "buy milk"}]


def test_stdin_is_read_when_the_text_is_a_dash(notes_module, store):
    code, out, _ = run(
        notes_module,
        ["add", "-", "--on", "2026-03-02", "--store", str(store)],
        stdin_text="piped in\n",
    )
    assert code == 0 and out == "added note 1\n"
    assert json.loads(store.read_text())["notes"][0]["text"] == "piped in"


def test_results_go_to_stdout_and_diagnostics_to_stderr(notes_module, store):
    run(notes_module, ["add", "one", "--on", "2026-03-01", "--store", str(store)])
    code, out, err = run(notes_module, ["list", "--format", "json", "-v", "--store", str(store)])
    assert code == 0
    assert json.loads(out)  # the RESULT parses as JSON, undisturbed...
    assert "1 note(s) from" in err  # ...because the chatter went elsewhere


def test_quiet_silences_success_but_not_the_work(notes_module, store):
    code, out, err = run(
        notes_module, ["add", "silent", "--on", "2026-03-01", "-q", "--store", str(store)]
    )
    assert (code, out, err) == (0, "", "")
    assert json.loads(store.read_text())["notes"][0]["text"] == "silent"


def test_a_refusal_exits_1_and_says_so_on_stderr(notes_module, store):
    store.write_text('{"version": 1, "notes": []}\n')
    code, out, err = run(notes_module, ["remove", "99", "--store", str(store)])
    assert code == 1
    assert out == ""
    assert "no note with id 99" in err


def test_search_exits_1_when_nothing_matched(notes_module, store):
    run(notes_module, ["add", "hello", "--on", "2026-03-01", "--store", str(store)])
    assert run(notes_module, ["search", "hello", "--store", str(store)])[0] == 0
    assert run(notes_module, ["search", "kangaroo", "--store", str(store)])[0] == 1


def test_ignore_case_changes_the_match(notes_module, store):
    run(notes_module, ["add", "Hello", "--on", "2026-03-01", "--store", str(store)])
    assert run(notes_module, ["search", "hello", "--store", str(store)])[0] == 1
    assert run(notes_module, ["search", "hello", "-i", "--store", str(store)])[0] == 0


def test_dry_run_leaves_the_store_byte_identical(notes_module, store):
    run(notes_module, ["add", "keep me", "--on", "2026-03-01", "--store", str(store)])
    before = store.read_bytes()

    code, out, err = run(notes_module, ["remove", "1", "--dry-run", "--store", str(store)])

    assert code == 0
    assert out == "would remove note 1\n"  # the RESULT: pipeable ids
    assert "was not touched" in err  # the DIAGNOSTIC
    assert store.read_bytes() == before  # the promise, actually checked


def test_a_real_remove_does_change_the_store(notes_module, store):
    """The control for the test above: without --dry-run, the bytes move."""
    run(notes_module, ["add", "keep me", "--on", "2026-03-01", "--store", str(store)])
    before = store.read_bytes()
    assert run(notes_module, ["remove", "1", "--store", str(store)])[0] == 0
    assert store.read_bytes() != before


def test_store_precedence_flag_beats_environment(notes_module, tmp_path, monkeypatch):
    from_flag = tmp_path / "flag.json"
    from_env = tmp_path / "env.json"
    monkeypatch.setenv("NOTES_STORE", str(from_env))

    run(notes_module, ["add", "a", "--on", "2026-03-01", "--store", str(from_flag)])
    assert from_flag.exists() and not from_env.exists()

    run(notes_module, ["add", "b", "--on", "2026-03-01"])
    assert from_env.exists()


def test_export_csv_has_a_header_and_one_row_per_note(notes_module, store):
    run(notes_module, ["add", "one", "-t", "x", "--on", "2026-03-01", "--store", str(store)])
    run(notes_module, ["add", "two", "--on", "2026-03-02", "--store", str(store)])
    code, out, _ = run(notes_module, ["export", "--format", "csv", "--store", str(store)])
    assert code == 0
    assert out.splitlines() == [
        "id,date,tags,text",
        "1,2026-03-01,x,one",
        "2,2026-03-02,,two",
    ]


def test_a_corrupt_store_is_a_refusal_not_a_traceback(notes_module, store):
    store.write_text("{ this is not json")
    code, out, err = run(notes_module, ["list", "--store", str(store)])
    assert code == 1 and out == "" and "not valid JSON" in err

Troubleshooting

Troubleshooting — Day 080 lab

NotImplementedError: EXERCISE 3: add the storage group ...

Working as designed. Each unfinished exercise raises this the moment its code would have run, so you always know exactly which one you are on. The test suite counts the remaining ones and tells you:

..  6 exercise(s) still unfinished — the behavioural battery
    will run against starter/notes.py once they are done.

Six raises cover eight exercises, because exercises 4, 5 and 6 all live inside build_parser and share one.

AttributeError: 'Namespace' object has no attribute 'func'

You forgot set_defaults(func=...) on a subparser, or the user ran a command with no subcommand at all and your add_subparsers call is missing required=True.

The second case is the interesting one. Without required=True, running plain notes parses successfully — argparse is happy, there simply is no subcommand — and then args.func explodes. With it, argparse refuses the command line, prints usage on standard error, and exits 2, which is what a user deserves. One keyword turns a crash into a usage message.

argparse.ArgumentError: argument -h/--help: conflicting option string

Your parent parser was built without add_help=False. A parser gets -h by default; when it is used as a parent, the child inherits that -h and then tries to add its own. Fix it in one place:

parent = argparse.ArgumentParser(add_help=False)

error: unrecognized arguments: --store /some/path

You put a shared option before the subcommand — notes --store x add "hi" — but the option is defined on the parent, which the subparsers inherit, not on the root parser. In this lab's design, shared options go after the subcommand: notes add "hi" --store x.

This is a real design decision, not a bug in argparse, and both conventions exist in the wild (git --no-pager log versus docker run --rm). Pick one and be consistent. If you want an option to work in both positions, add it to the root parser and the parent — and be aware that the later one wins, which surprises people.

The --on value arrives as a string, not a date

You passed type=iso_date as a string: type="iso_date". It must be the function object itself, with no quotes and no call parentheses:

add.add_argument("--on", type=iso_date)      # right
add.add_argument("--on", type=iso_date())    # wrong: calls it immediately
add.add_argument("--on", type="iso_date")    # wrong: argparse cannot use a string

My error message shows a full Python traceback

You raised ValueError (or let one escape) instead of argparse.ArgumentTypeError. Only ArgumentTypeError — and TypeError and ValueError raised inside a type= callable — get converted into a usage message. An exception raised in your handler is not converted at all: catch it and turn it into a NotesError, which main() already renders as one tidy line on standard error with exit 1.

notes add - just sits there doing nothing

Standard input is a terminal and the program is waiting for you to type. Press Ctrl-D (on a line of its own) to signal end-of-file, or Ctrl-C to give up.

That hang is exactly what exercise 7's terminal detection prevents. Once it is implemented, notes add - with nothing piped in exits 2 with a message instead of hanging. Try both:

notes add -                      # detected: exits 2 with an explanation
echo "piped in" | notes add -    # reads from the pipe, exits 0

--dry-run prints the right thing but the test still fails

Read the failing check's name. There are two different ones:

  • "leaves the store BYTE-IDENTICAL" — your code fell through to save_store. A dry run must return 0 before the write. Re-indent, or add the missing return.
  • "puts the ids on stdout, where they can be piped" — you wrote both lines to the same stream. The ids are a result (standard output); the summary is a diagnostic (standard error).

The dry-run and real-remove output arrive in the wrong order

You wrote to standard output and standard error without flushing in between. When standard output is a terminal Python line-buffers it, so the order looks right; the moment you redirect to a file it becomes block-buffered and the unbuffered standard error overtakes it. Call streams.stdout.flush() before writing the diagnostic. This is not cosmetic — it is why captured logs from build servers so often look scrambled.

pytest: command not found

The suite tells you what to do, but for completeness:

python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt

Or point it at an installation you already have:

PYTEST=/path/to/pytest bash tests/run_tests.sh

ModuleNotFoundError: No module named 'notes' from pytest

tests/conftest.py puts the right directory on sys.path, chosen by the NOTES_DIR environment variable. Run pytest from the lab directory, not from inside tests/:

cd labs/sections/programming-with-python/day-080-building-clis-with-argparse
NOTES_DIR=examples pytest tests -q     # the reference
NOTES_DIR=starter  pytest tests -q     # your work

notes list --format json | head -2 prints a BrokenPipeError

head closes the pipe as soon as it has its two lines, and the writer finds out the hard way. main() already catches BrokenPipeError and returns 0, because a downstream program deciding it has seen enough is not your program's failure. If you are writing your own tool and see this, catch it in the same place — do not sprinkle try/except through the renderers.

Everything passes but the counts differ from expected-output/test-run.txt

test-run.txt was captured with the starter unfinished: 76 checks. With all eight exercises done the battery runs a second time against your file and the total is 133 checks. Both are correct; only 0 failure(s). matters.

Security notes

Security notes — Day 080 lab

A command-line tool is a program that takes input from strangers. Sometimes the stranger is you in six months; sometimes it is a shell script running as a scheduled job with arguments assembled from somewhere else. The interface is a trust boundary, and argparse is the first place you can defend it.

What this lab does, and does not do

  • No network. Nothing here opens a socket, resolves a name, or downloads anything. The only install is pytest, and that is optional if you already have it.
  • No credentials, no keys, no accounts. The tool stores plain notes in a plain JSON file.
  • Writes only where you point it. The store path comes from --store, then $NOTES_STORE, then ./notes.json in the current directory. The test suite writes exclusively inside a mktemp -d directory and removes it in a trap, so a failed run leaves nothing behind.

Validation at the boundary is a security control

Every type= function in this tool is a gate. iso_date means no code past the parser has to wonder whether args.on is a date — it is a datetime.date or the program already exited 2. positive_int means no handler receives a negative note id. choices=["table", "json"] means the format selector can never hold a third value.

This matters more than it looks. The classic command-line vulnerability is a value that is accepted at the edge, carried untouched through three layers, and finally interpolated somewhere dangerous. Converting and validating at the parser — the moment the value enters — means the dangerous shape never exists inside the program at all. It is the same argument Day 70 made for validating in __post_init__, moved out to the process boundary.

Never build a shell command out of user input

This tool does not run subprocesses, but the day you write one that does, the rule is absolute:

subprocess.run(["grep", pattern, path])          # a list: safe
subprocess.run(f"grep {pattern} {path}", shell=True)   # never do this

With shell=True a pattern of x; rm -rf ~ is not a pattern, it is two commands. Passing a list means the operating system hands your arguments to the program directly and no shell ever sees them. Day 81 uses subprocess properly; this is the habit to bring to it.

Path arguments deserve suspicion

--store takes a path, and a path from a stranger can point anywhere: ../../.ssh/config, a symbolic link, a device file. This lab's tool is a personal note keeper, so it does what a personal tool should — writes where you say. A tool that accepts a path from a less trusted source should additionally:

  • resolve it with Path(value).resolve() and confirm it is inside an expected directory before opening it;
  • refuse symbolic links where they are not wanted (Path.is_symlink());
  • open with the narrowest mode that works, and set restrictive permissions on files it creates.

--dry-run is a safety feature, not a convenience

Any command that deletes, overwrites, sends, or spends should have one, and it should be exercised in tests the way this lab's is — by hashing the target before and after. A --dry-run that has never been checked is a claim, not a guarantee, and it is the kind of claim people trust right up until the moment it turns out to be false.

Consider also a confirmation prompt for genuinely destructive operations, with a --yes/--force flag to skip it in scripts. The pattern is: interactive users get a chance to stop; automation opts out explicitly and on the record.

Secrets do not belong on the command line

A password or API key passed as --token abc123 is visible to every user on the machine through the process list, and lands in your shell history file in plain text. The conventions that avoid this:

  • read the secret from an environment variable (os.environ["API_TOKEN"]);
  • read it from a file whose path is the argument, not the secret itself (--token-file);
  • read it from standard input, which is exactly the - convention this lab implements for note text.

The add - exercise is therefore not only about pipelines. It is the same mechanism you use to keep a secret off the command line.

JSON, not pickle

The store is read with json.loads, which builds only lists, dicts, strings, numbers, booleans and null — it cannot execute anything. pickle can and does execute arbitrary code while loading, so a pickle file is as dangerous as a program. A malformed store here produces notes: ... is not valid JSON and exit 1; a malicious pickle would produce whatever its author wanted.

The tool also checks the shape of what it loaded, not just that it parsed, and refuses a file that is valid JSON but not a notes store. Parsing and validating are two different steps.

Errors on standard error, always

Sending diagnostics to standard error is a correctness rule, but it has a security edge: a pipeline like notes export --format json | some-importer must never have an error message injected into the middle of its data. Keeping the two streams separate means a downstream program sees either good data or no data, and never a plausible-looking mixture of the two.

Personal data

Notes are personal data. This store is a plain file in your working directory with no encryption; treat it accordingly, and keep it out of version control. A .gitignore entry for notes.json costs nothing.