Capstone › Capstone Build II: Ship It › Day 363
Hands-on lab — Day 363: Portfolio, Resume, and Sharing Your Work
- ← Back to the Day 363 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/capstone/day-363-portfolio-resume-and-sharing-your-work/
Commands
Setup
pip install -r requirements/requirements.txt Run
python examples/claims_demo.py Test
bash tests/run_tests.sh File tree
examples/claims_demo.py examples/claims.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/claims.py tests/run_tests.sh tests/test_claims.py troubleshooting.md
Lab README
Lab — Day 363: Portfolio, Resume, and Sharing Your Work
Lesson
- Lesson title: Portfolio, Resume, and Sharing Your Work
- Day number: 363 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-363-portfolio-resume-and-sharing-your-work
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-363-portfolio-resume-and-sharing-your-workwhen the site is running.
Purpose
Build a checker for the claims in a portfolio: is each one specific, does it name a baseline, does it say who did the work, and does it link to something a stranger can open.
The checker does not judge whether a claim is true. It judges whether it is checkable, which is the property a reader can assess without knowing you.
Learning objectives
- Detect a measurement, including a stated change between two figures with no unit.
- Detect a baseline, and explain why a figure without one is not a result.
- Classify attribution as yours, shared, mixed or unattributed.
- Decide whether an evidence link is openable by a reader, without a network.
- Produce a rewrite instruction rather than only a grade.
Prerequisites
- Day 362, "Documentation and Demo".
- Comfortable with Python regular expressions, dataclasses and enums.
Supported operating systems
macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.
Hardware requirements
Any machine running Python 3.10 or newer. No GPU. The suite completes in hundredths of a second.
Required software
- Python 3.10 or newer.
pytest7 or newer.
Free and open-source options
Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. No account, no API key, no network — evidence links are judged from the string rather than by fetching them, which keeps the check fast and offline.
Installation
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
Removing .venv fully reverses the installation.
File structure
starter/claims.py your work: five detection and grading functions
examples/claims.py reference implementation
examples/claims_demo.py assesses five claims and shows one rewritten
tests/test_claims.py grouped by property
tests/run_tests.sh suite entry point
expected-output/ real captured output and measured values
requirements/ pinned dependency
How to run
python3 examples/claims_demo.py # assess a set of claims
bash tests/run_tests.sh # run the suite
To work on the exercise, edit starter/claims.py, then copy it over the reference:
cp starter/claims.py examples/claims.py
bash tests/run_tests.sh
Keep a copy of the reference first if you want to compare.
What the commands do
python3 examples/claims_demo.py assesses five portfolio claims ranging from vague to strong, prints what each weak one is missing as an instruction, then shows one claim written badly and written well.
bash tests/run_tests.sh runs pytest over nineteen tests grouped by property: measurement, baseline, attribution, evidence, grading, hints and the report.
Expected output
VAGUE Significantly improved retrieval quality.
vague wording: significantly; no measurement; does not say who did it;
no evidence link
WEAK Reduced answer latency to 840ms.
measurement without a baseline; does not say who did it
STRONG I cut p95 answer latency from 4.2s to 840ms by adding an
=> strong=1 weak=2 vague=2
--- the same claim, before and after ---
VAGUE Significantly improved retrieval quality.
STRONG I raised recall@10 from 0.71 to 0.94 by adding a reranking stage,
measured over 200 held-out questions.
expected-output/FIELDS.md explains the grades and why WEAK and VAGUE are distinguished.
Validation steps
bash tests/run_tests.shreports19 passed.- "Reduced answer latency to 840ms" must grade
WEAK, notSTRONG. A number alone is not a result. - "from 0.71 to 0.94" must count as a measurement despite having no unit.
- A claim with a vague word and a measurement must be
WEAK, notVAGUE— vague is reserved for claims with nothing measured to fall back on.
Tests
Nineteen tests in seven groups:
- measurement — units, percentages and multipliers count; a stated change between two figures counts without a unit; adjectives do not; hyphenated vague words are caught.
- baseline — "from", "previously" and "compared to" are recognised; an end value alone is not.
- attribution — mine, shared and unattributed are classified; mixed is detected; the match is case-insensitive.
- evidence — a public URL is openable; empty,
localhost, a loopback IP,file://and bare paths are not. - grading — a complete claim is strong with no reasons; an adjective with no measurement is vague; a measurement without a baseline is weak; every missing property is named.
- hints — a strong claim needs nothing; a weak one is told what to add.
- report — each grade is counted.
Cleanup
rm -rf __pycache__ .pytest_cache
rm -rf .venv
Both are safe to remove and fully reversible.
Troubleshooting
See troubleshooting.md.
Security notes
See security.md. In short: no network, no credentials — and the lesson covers a portfolio as a permanent public disclosure.
Extension exercises
- A verb check. Flag claims that describe a position ("was responsible for") rather than an action ("rebuilt, cutting p95 from 2.1s to 600ms").
- Run it on your own portfolio. Record the grade distribution before and after rewriting.
- Categorise what you cannot fix. For each claim that resists becoming strong, decide whether the missing piece is a measurement you never took, evidence an employer owns, or attribution you are reluctant to make precise.
Navigation
- Lesson
- Previous: Day 362 — Documentation and Demo
- Next: Day 364 — Capstone Retrospective
Expected output
FIELDS.md
# Reading the claim assessment
## An assessment line
WEAK Reduced answer latency to 840ms.
measurement without a baseline; does not say who did it
| Part | Meaning |
| --- | --- |
| grade | `STRONG`, `WEAK` or `VAGUE`. |
| claim | The sentence, truncated for display. |
| reasons | Every property that is missing, so the fix is obvious. Absent on a strong claim. |
## The three grades
| Grade | Meaning |
| --- | --- |
| `STRONG` | All four properties present — measured, with a baseline, attributed, and linked to something openable. |
| `WEAK` | Something is missing, and it is fixable by adding words. |
| `VAGUE` | Vague wording **and** no measurement. The weakest kind: it sounds like a result and contains none. |
The distinction between `WEAK` and `VAGUE` is about what it costs to fix. A missing baseline is a sentence away. A claim with nothing measured needs you to go and measure something.
## The four properties
| Property | Failure it prevents |
| --- | --- |
| measurement | An adjective standing where a figure should be. |
| baseline | A number that describes the present rather than a change. |
| attribution | A reader unable to tell what you did from what the team did. |
| evidence | A link only the author can open — a loopback address, a `file://` path, a bare filesystem path. |
## The hint lines
Reduced answer latency to 840ms. add the value it started from, what you personally did
A grade says something is wrong. A hint says what to add. The second is the one you can act on, which is why the report produces both.
examples-run.txt
--- claims as written ---
VAGUE Built a state-of-the-art RAG system with blazing perform
vague wording: blazing, state-of-the-art; no measurement; does not say who did it; no evidence link
VAGUE Significantly improved retrieval quality.
vague wording: significantly; no measurement; does not say who did it; no evidence link
WEAK Reduced answer latency to 840ms.
measurement without a baseline; does not say who did it
STRONG I cut p95 answer latency from 4.2s to 840ms by adding an
WEAK We shipped a support assistant used by 200 people.
measurement without a baseline; evidence link is not openable by a reader
=> strong=1 weak=2 vague=2
--- what each weak claim needs ---
Built a state-of-the-art RAG system with bla add a number with a unit, what you personally did, a link a stranger can open, the adjective replaced by the figure
Significantly improved retrieval quality. add a number with a unit, what you personally did, a link a stranger can open, the adjective replaced by the figure
Reduced answer latency to 840ms. add the value it started from, what you personally did
We shipped a support assistant used by 200 p add the value it started from, a link a stranger can open
--- the same claim, before and after ---
VAGUE Significantly improved retrieval quality.
STRONG I raised recall@10 from 0.71 to 0.94 by adding a reranking stage, measured over 200 held-out questions.
measured-values.txt
Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.
python3 examples/claims_demo.py -> 20 lines, exit 0
bash tests/run_tests.sh -> 19 passed in 0.01s
Five claims: strong=1 weak=2 vague=2.
The two worth studying:
"Reduced answer latency to 840ms." WEAK
measurement without a baseline; does not say who did it
This one SOUNDS rigorous. There is a number and a unit. It is still
unassessable, because 840ms might be an improvement from 4.2s or a
regression from 200ms.
"We shipped a support assistant used by 200 people." WEAK
measurement without a baseline; evidence link is not openable
Measured and attributed, and its evidence link points at a loopback address --
evidence to the author and to nobody else, so it functions as no link.
The rewrite, same work described twice:
VAGUE Significantly improved retrieval quality.
STRONG I raised recall@10 from 0.71 to 0.94 by adding a reranking stage,
measured over 200 held-out questions.
The second is longer and none of it is padding: what changed, from what to
what, by what mechanism, over what sample, done by whom.
One implementation note. "from 0.71 to 0.94" has no unit and IS a measurement.
An earlier version of this checker required a unit and graded that rewrite WEAK
-- a checker that would have pushed writers into padding sentences with units
that do not fit the quantity.
run-output.txt
--- claims as written ---
VAGUE Built a state-of-the-art RAG system with blazing perform
vague wording: blazing, state-of-the-art; no measurement; does not say who did it; no evidence link
VAGUE Significantly improved retrieval quality.
vague wording: significantly; no measurement; does not say who did it; no evidence link
WEAK Reduced answer latency to 840ms.
measurement without a baseline; does not say who did it
STRONG I cut p95 answer latency from 4.2s to 840ms by adding an
WEAK We shipped a support assistant used by 200 people.
measurement without a baseline; evidence link is not openable by a reader
=> strong=1 weak=2 vague=2
--- what each weak claim needs ---
Built a state-of-the-art RAG system with bla add a number with a unit, what you personally did, a link a stranger can open, the adjective replaced by the figure
Significantly improved retrieval quality. add a number with a unit, what you personally did, a link a stranger can open, the adjective replaced by the figure
Reduced answer latency to 840ms. add the value it started from, what you personally did
We shipped a support assistant used by 200 p add the value it started from, a link a stranger can open
--- the same claim, before and after ---
VAGUE Significantly improved retrieval quality.
STRONG I raised recall@10 from 0.71 to 0.94 by adding a reranking stage, measured over 200 held-out questions.
starter-run.txt
An untouched starter fails nearly the whole suite:
18 failed, 1 passed in 0.05s
The one that passes is test_vague_words_are_listed_including_hyphenated_forms,
which exercises vague_words_in -- not one of the stubbed functions. Everything
else depends on has_measurement, has_baseline, attribution, is_openable or
assess.
test-output.txt
................... [100%]
19 passed in 0.01s
test-run.txt
................... [100%]
19 passed in 0.01s
Source files
examples/claims_demo.py (1703 bytes)
#!/usr/bin/env python3
"""Assess a set of portfolio claims, then show the same claim rewritten."""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from claims import Claim, review, rewrite_hint
CLAIMS = [
Claim("Built a state-of-the-art RAG system with blazing performance."),
Claim("Significantly improved retrieval quality."),
Claim("Reduced answer latency to 840ms.", "https://example.com/writeup"),
Claim(
"I cut p95 answer latency from 4.2s to 840ms by adding an IVF index, "
"while the team migrated the ingestion pipeline.",
"https://example.com/capstone",
),
Claim("We shipped a support assistant used by 200 people.", "http://127.0.0.1:8080"),
]
WEAK_THEN_STRONG = (
Claim("Significantly improved retrieval quality."),
Claim(
"I raised recall@10 from 0.71 to 0.94 by adding a reranking stage, "
"measured over 200 held-out questions.",
"https://example.com/eval",
),
)
def main() -> int:
report = review(CLAIMS)
print("--- claims as written ---")
for assessment in report.assessments:
print(assessment.line())
print(f" => {report.summary()}")
print("--- what each weak claim needs ---")
for claim in CLAIMS:
hint = rewrite_hint(claim)
if hint != "nothing to add":
print(f" {claim.text[:44]:<46} {hint}")
print("--- the same claim, before and after ---")
for claim in WEAK_THEN_STRONG:
a = review([claim]).assessments[0]
print(f" {a.grade.value.upper():<7} {claim.text}")
return 0
if __name__ == "__main__":
raise SystemExit(main())
examples/claims.py (7217 bytes)
"""Check that the claims in a portfolio are specific, attributed and evidenced.
Offline and standard-library only. The subject is a list of written claims, so
nothing is fetched and nothing is executed.
A portfolio is a set of claims about what you did. A reader -- a hiring
manager, a collaborator, a reviewer -- is deciding how much to believe, and
they do that on four properties that are visible in the sentence itself:
specific a number with a unit beats an adjective
baseline "reduced to 840ms" is meaningless without "from 4.2s"
attributed what YOU did, distinct from what the team did
evidenced a link to something a reader can actually open
The uncomfortable one is attribution. Writing "I" where the work was shared is
dishonest; writing "we" for work that was yours undersells it. Both are common,
and the fix is the same -- say who did what, precisely.
Nothing here judges whether a claim is TRUE. It checks whether a claim is
CHECKABLE, which is the property a reader can assess without knowing you.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from enum import Enum
class Grade(str, Enum):
STRONG = "strong"
WEAK = "weak"
VAGUE = "vague"
# Adjectives that sound like measurements and are not.
VAGUE_WORDS = (
"significantly",
"greatly",
"massively",
"dramatically",
"substantially",
"much faster",
"much better",
"world-class",
"cutting-edge",
"state of the art",
"state-of-the-art",
"robust",
"seamless",
"blazing",
)
# What counts as a measurement. Three shapes, because real claims use all
# three and a checker that only accepts one pushes people toward padding their
# sentences with units that do not belong.
MEASUREMENT = re.compile(
# a number with a unit, a percentage, a multiplier, or a currency amount
r"\b\d+(?:\.\d+)?\s?(?:%|x\b|ms|s\b|kb|mb|gb|k\b|m\b|hours?|days?|weeks?|months?"
r"|people|users|requests|documents|questions|queries)"
r"|[$£€]\s?\d+(?:[.,]\d+)?"
# a stated change between two figures -- "from 0.71 to 0.94" is a
# measurement even though a ratio has no unit
r"|\bfrom\s+\d+(?:\.\d+)?\s*\w*\s+to\s+\d+(?:\.\d+)?",
re.I,
)
# Wording that names a starting point, so a change can be judged.
BASELINE = re.compile(r"\bfrom\b|\bbaseline\b|\bpreviously\b|\bwas\b|\bcompared (?:to|with)\b", re.I)
# First-person singular, and shared-credit wording.
MINE = re.compile(r"\bI\b|\bmy\b", re.I)
OURS = re.compile(r"\bwe\b|\bour\b|\bteam\b", re.I)
# Addresses that resolve only on the author's own machine. A loopback IP is
# exactly as unopenable as the name it usually stands for, and people write
# both, so both belong here.
LOCAL_PREFIXES = (
"http://localhost",
"https://localhost",
"http://127.0.0.1",
"https://127.0.0.1",
"http://[::1]",
"https://[::1]",
"file://",
"/",
)
@dataclass(frozen=True)
class Claim:
text: str
evidence_url: str = ""
@dataclass
class Assessment:
claim: str
grade: Grade
reasons: list[str] = field(default_factory=list)
def line(self) -> str:
head = f" {self.grade.value.upper():<7} {self.claim[:56]}"
return f"{head}\n {'; '.join(self.reasons)}" if self.reasons else head
@dataclass
class PortfolioReport:
assessments: list[Assessment] = field(default_factory=list)
def count(self, grade: Grade) -> int:
return sum(1 for a in self.assessments if a.grade is grade)
def summary(self) -> str:
return (
f"strong={self.count(Grade.STRONG)} weak={self.count(Grade.WEAK)} "
f"vague={self.count(Grade.VAGUE)}"
)
def has_measurement(text: str) -> bool:
return MEASUREMENT.search(text) is not None
def vague_words_in(text: str) -> list[str]:
lowered = text.lower()
return sorted(w for w in VAGUE_WORDS if w in lowered)
def has_baseline(text: str) -> bool:
"""Whether the claim names a starting point.
A change without a baseline is not a measurement. "Reduced latency to
840ms" could be an improvement, a regression, or unchanged.
"""
return BASELINE.search(text) is not None
def attribution(text: str) -> str:
"""`mine`, `shared`, `mixed` or `unattributed`.
`mixed` -- both first-person singular and team wording in one claim -- is
the strongest form, because it separates what you did from what the group
did. It is also the rarest, because it takes more words.
"""
mine, ours = bool(MINE.search(text)), bool(OURS.search(text))
if mine and ours:
return "mixed"
if mine:
return "mine"
if ours:
return "shared"
return "unattributed"
def is_openable(url: str) -> bool:
"""Whether a reader could actually open this.
Deliberately shallow: no network. A local path or a private host is not
evidence to someone outside your machine or your company, and that is
decidable from the string alone.
"""
if not url:
return False
if url.startswith(LOCAL_PREFIXES):
return False
return url.startswith(("http://", "https://"))
def assess(claim: Claim) -> Assessment:
"""Grade one claim on the four properties."""
reasons: list[str] = []
vague = vague_words_in(claim.text)
measured = has_measurement(claim.text)
baseline = has_baseline(claim.text)
who = attribution(claim.text)
evidenced = is_openable(claim.evidence_url)
if vague:
reasons.append(f"vague wording: {', '.join(vague)}")
if not measured:
reasons.append("no measurement")
elif not baseline:
reasons.append("measurement without a baseline")
if who == "unattributed":
reasons.append("does not say who did it")
if not evidenced:
reasons.append(
"evidence link is not openable by a reader" if claim.evidence_url else "no evidence link"
)
# A claim with no measurement AND vague wording is the weakest kind: it
# sounds like a result and contains none.
if vague and not measured:
grade = Grade.VAGUE
elif not reasons:
grade = Grade.STRONG
else:
grade = Grade.WEAK
return Assessment(claim.text, grade, reasons)
def review(claims: list[Claim]) -> PortfolioReport:
return PortfolioReport(assessments=[assess(c) for c in claims])
def rewrite_hint(claim: Claim) -> str:
"""What this specific claim is missing, as a sentence to act on."""
a = assess(claim)
if a.grade is Grade.STRONG:
return "nothing to add"
wants = []
if "no measurement" in a.reasons:
wants.append("a number with a unit")
if "measurement without a baseline" in a.reasons:
wants.append("the value it started from")
if "does not say who did it" in a.reasons:
wants.append("what you personally did")
if any(r.startswith("no evidence") or r.startswith("evidence link") for r in a.reasons):
wants.append("a link a stranger can open")
if any(r.startswith("vague wording") for r in a.reasons):
wants.append("the adjective replaced by the figure")
return "add " + ", ".join(wants) if wants else "nothing to add"
metadata.yml (836 bytes)
lesson_id: D363
day: 363
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/claims_demo.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3 examples/claims_demo.py
reproduced the documented assessments exactly (strong=1 weak=2 vague=2); bash tests/run_tests.sh -> 19
passed in 0.01s. The untouched starter fails 18 of 19; the 1 that passes exercises vague_words_in,
which is not stubbed. requires_network is false: evidence links are judged from the string rather than
fetched, so the check is offline by design.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/claims.py (7697 bytes)
"""Check that the claims in a portfolio are specific, attributed and evidenced.
Offline and standard-library only. The subject is a list of written claims, so
nothing is fetched and nothing is executed.
A portfolio is a set of claims about what you did. A reader -- a hiring
manager, a collaborator, a reviewer -- is deciding how much to believe, and
they do that on four properties that are visible in the sentence itself:
specific a number with a unit beats an adjective
baseline "reduced to 840ms" is meaningless without "from 4.2s"
attributed what YOU did, distinct from what the team did
evidenced a link to something a reader can actually open
The uncomfortable one is attribution. Writing "I" where the work was shared is
dishonest; writing "we" for work that was yours undersells it. Both are common,
and the fix is the same -- say who did what, precisely.
Nothing here judges whether a claim is TRUE. It checks whether a claim is
CHECKABLE, which is the property a reader can assess without knowing you.
"""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from enum import Enum
class Grade(str, Enum):
STRONG = "strong"
WEAK = "weak"
VAGUE = "vague"
# Adjectives that sound like measurements and are not.
VAGUE_WORDS = (
"significantly",
"greatly",
"massively",
"dramatically",
"substantially",
"much faster",
"much better",
"world-class",
"cutting-edge",
"state of the art",
"state-of-the-art",
"robust",
"seamless",
"blazing",
)
# What counts as a measurement. Three shapes, because real claims use all
# three and a checker that only accepts one pushes people toward padding their
# sentences with units that do not belong.
MEASUREMENT = re.compile(
# a number with a unit, a percentage, a multiplier, or a currency amount
r"\b\d+(?:\.\d+)?\s?(?:%|x\b|ms|s\b|kb|mb|gb|k\b|m\b|hours?|days?|weeks?|months?"
r"|people|users|requests|documents|questions|queries)"
r"|[$£€]\s?\d+(?:[.,]\d+)?"
# a stated change between two figures -- "from 0.71 to 0.94" is a
# measurement even though a ratio has no unit
r"|\bfrom\s+\d+(?:\.\d+)?\s*\w*\s+to\s+\d+(?:\.\d+)?",
re.I,
)
# Wording that names a starting point, so a change can be judged.
BASELINE = re.compile(r"\bfrom\b|\bbaseline\b|\bpreviously\b|\bwas\b|\bcompared (?:to|with)\b", re.I)
# First-person singular, and shared-credit wording.
MINE = re.compile(r"\bI\b|\bmy\b", re.I)
OURS = re.compile(r"\bwe\b|\bour\b|\bteam\b", re.I)
# Addresses that resolve only on the author's own machine. A loopback IP is
# exactly as unopenable as the name it usually stands for, and people write
# both, so both belong here.
LOCAL_PREFIXES = (
"http://localhost",
"https://localhost",
"http://127.0.0.1",
"https://127.0.0.1",
"http://[::1]",
"https://[::1]",
"file://",
"/",
)
@dataclass(frozen=True)
class Claim:
text: str
evidence_url: str = ""
@dataclass
class Assessment:
claim: str
grade: Grade
reasons: list[str] = field(default_factory=list)
def line(self) -> str:
head = f" {self.grade.value.upper():<7} {self.claim[:56]}"
return f"{head}\n {'; '.join(self.reasons)}" if self.reasons else head
@dataclass
class PortfolioReport:
assessments: list[Assessment] = field(default_factory=list)
def count(self, grade: Grade) -> int:
return sum(1 for a in self.assessments if a.grade is grade)
def summary(self) -> str:
return (
f"strong={self.count(Grade.STRONG)} weak={self.count(Grade.WEAK)} "
f"vague={self.count(Grade.VAGUE)}"
)
def has_measurement(text: str) -> bool:
# TASK 1: True when the text contains a real quantity.
# Three shapes count: a number with a unit or percentage or multiplier, a
# currency amount, and a STATED CHANGE between two figures ("from 0.71 to
# 0.94"). The third matters: a ratio is a measurement without a unit, and
# demanding one pushes writers into padding sentences.
raise NotImplementedError("implement has_measurement")
def vague_words_in(text: str) -> list[str]:
lowered = text.lower()
return sorted(w for w in VAGUE_WORDS if w in lowered)
def has_baseline(text: str) -> bool:
"""Whether the claim names a starting point.
A change without a baseline is not a measurement. "Reduced latency to
840ms" could be an improvement, a regression, or unchanged.
"""
# TASK 2: True when the claim names a starting point -- "from",
# "previously", "was", "compared to". A figure without one describes the
# present rather than a change.
raise NotImplementedError("implement has_baseline")
def attribution(text: str) -> str:
"""`mine`, `shared`, `mixed` or `unattributed`.
`mixed` -- both first-person singular and team wording in one claim -- is
the strongest form, because it separates what you did from what the group
did. It is also the rarest, because it takes more words.
"""
# TASK 3: return "mine", "shared", "mixed" or "unattributed".
# Both singular and team wording present is "mixed", and it is the
# STRONGEST form -- it separates your work from the group's, which is the
# question the reader actually has.
raise NotImplementedError("implement attribution")
def is_openable(url: str) -> bool:
"""Whether a reader could actually open this.
Deliberately shallow: no network. A local path or a private host is not
evidence to someone outside your machine or your company, and that is
decidable from the string alone.
"""
# TASK 4: True only for an http(s) URL a stranger could follow.
# Empty, a local address, file:// and a bare absolute path are all evidence to
# the author alone. No network: this is decidable from the string.
raise NotImplementedError("implement is_openable")
def assess(claim: Claim) -> Assessment:
"""Grade one claim on the four properties."""
# TASK 5: grade one claim, collecting a reason for each missing property.
# vague wording -> name the words found
# no measurement -> "no measurement"
# measured, no baseline-> "measurement without a baseline"
# unattributed -> "does not say who did it"
# no openable evidence -> "no evidence link" / "evidence link is not
# openable by a reader"
# Grade VAGUE only when there is vague wording AND no measurement -- that
# is the claim that sounds like a result and contains none. STRONG when
# there are no reasons at all. Otherwise WEAK, which means fixable.
raise NotImplementedError("implement assess")
def review(claims: list[Claim]) -> PortfolioReport:
return PortfolioReport(assessments=[assess(c) for c in claims])
def rewrite_hint(claim: Claim) -> str:
"""What this specific claim is missing, as a sentence to act on."""
a = assess(claim)
if a.grade is Grade.STRONG:
return "nothing to add"
wants = []
if "no measurement" in a.reasons:
wants.append("a number with a unit")
if "measurement without a baseline" in a.reasons:
wants.append("the value it started from")
if "does not say who did it" in a.reasons:
wants.append("what you personally did")
if any(r.startswith("no evidence") or r.startswith("evidence link") for r in a.reasons):
wants.append("a link a stranger can open")
if any(r.startswith("vague wording") for r in a.reasons):
wants.append("the adjective replaced by the figure")
return "add " + ", ".join(wants) if wants else "nothing to add"
tests/run_tests.sh (205 bytes)
#!/usr/bin/env bash
# Run the day 363 portfolio claim test suite.
# Offline, standard library plus pytest. No network, no API key.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_claims.py (4999 bytes)
"""Grouped by property, so a failure names which one stopped being detected.
Run with: bash tests/run_tests.sh
"""
from __future__ import annotations
import os
import sys
HERE = os.path.dirname(os.path.abspath(__file__))
LAB = os.path.dirname(HERE)
sys.path.insert(0, os.path.join(LAB, "examples"))
from claims import ( # noqa: E402
Claim,
Grade,
assess,
attribution,
has_baseline,
has_measurement,
is_openable,
review,
rewrite_hint,
vague_words_in,
)
STRONG = Claim(
"I cut p95 answer latency from 4.2s to 840ms by adding an IVF index, "
"while the team migrated the ingestion pipeline.",
"https://example.com/capstone",
)
# ------------------------------------------------------------- measurement
def test_units_percentages_and_multipliers_count():
for text in ("840ms", "reduced by 40%", "3x faster", "12 hours", "$400"):
assert has_measurement(text), text
def test_a_stated_change_between_two_figures_counts_without_a_unit():
# recall 0.71 -> 0.94 is a measurement. A checker that demands a unit
# pushes people into padding sentences with units that do not belong.
assert has_measurement("raised recall@10 from 0.71 to 0.94")
def test_adjectives_are_not_measurements():
for text in ("much faster", "significantly better", "robust and seamless"):
assert not has_measurement(text), text
def test_vague_words_are_listed_including_hyphenated_forms():
assert vague_words_in("a state-of-the-art, blazing system") == ["blazing", "state-of-the-art"]
# ---------------------------------------------------------------- baseline
def test_a_baseline_is_recognised():
assert has_baseline("from 4.2s to 840ms")
assert has_baseline("previously 12 minutes")
assert has_baseline("compared to the old pipeline")
def test_an_end_value_alone_has_no_baseline():
# "Reduced latency to 840ms" could be an improvement, a regression, or no
# change at all. Without a starting point it is not a measurement.
assert not has_baseline("reduced answer latency to 840ms")
# ------------------------------------------------------------- attribution
def test_attribution_is_classified():
assert attribution("I built the index") == "mine"
assert attribution("We shipped it") == "shared"
assert attribution("The system was shipped") == "unattributed"
def test_mixed_attribution_separates_your_work_from_the_team_s():
assert attribution("I added the reranker while the team ran the migration") == "mixed"
def test_attribution_is_case_insensitive():
assert attribution("my contribution") == "mine"
assert attribution("OUR TEAM") == "shared"
# ---------------------------------------------------------------- evidence
def test_a_public_url_is_openable():
assert is_openable("https://example.com/writeup")
def test_local_and_missing_links_are_not_evidence():
# Not evidence to anyone but you, and that is decidable without a network.
for url in (
"",
"http://localhost",
"http://127.0.0.1:8080",
"https://[::1]",
"file:///home/me/notes.md",
"/home/me/x.md",
):
assert not is_openable(url), url
# ------------------------------------------------------------------ grading
def test_a_complete_claim_is_strong_with_no_reasons():
a = assess(STRONG)
assert a.grade is Grade.STRONG
assert a.reasons == []
def test_an_adjective_with_no_measurement_is_vague():
a = assess(Claim("Significantly improved retrieval quality."))
assert a.grade is Grade.VAGUE
assert any("vague wording" in r for r in a.reasons)
def test_a_measurement_without_a_baseline_is_weak_not_vague():
a = assess(Claim("Reduced answer latency to 840ms.", "https://example.com/x"))
assert a.grade is Grade.WEAK
assert "measurement without a baseline" in a.reasons
def test_a_measured_claim_with_a_vague_word_is_weak_not_vague():
# Vague only when there is nothing measured to fall back on.
a = assess(Claim("I significantly cut latency from 4.2s to 840ms.", "https://example.com/x"))
assert a.grade is Grade.WEAK
def test_every_missing_property_is_named():
a = assess(Claim("Built a robust system."))
assert len(a.reasons) == 4 # vague, no measurement, unattributed, no evidence
# -------------------------------------------------------------------- hints
def test_a_strong_claim_needs_nothing():
assert rewrite_hint(STRONG) == "nothing to add"
def test_the_hint_names_what_is_missing():
hint = rewrite_hint(Claim("Reduced answer latency to 840ms."))
assert "the value it started from" in hint
assert "what you personally did" in hint
# ------------------------------------------------------------------ report
def test_the_report_counts_each_grade():
report = review(
[STRONG, Claim("Significantly improved things."), Claim("Reduced latency to 840ms.")]
)
assert report.summary() == "strong=1 weak=1 vague=1"
Troubleshooting
Troubleshooting — Day 363
"from 0.71 to 0.94" is not detected as a measurement
Your pattern requires a unit. A ratio, a rate or a score is a legitimate measurement without one, and a stated change between two figures is the clearest form of all.
This lab had that bug: an earlier version graded the rewritten strong claim as WEAK, which would have pushed writers into padding sentences with units that do not fit the quantity. Add a case for from <number> ... to <number>.
"state-of-the-art" is not flagged
Your vague-word list has the spaced form only. Both spellings are common in the wild; include them both.
Every claim comes out VAGUE
You are grading VAGUE whenever a vague word appears. Vague requires both a vague word and no measurement — that is the claim which sounds like a result and contains none. A measured claim with a stray adjective is WEAK, because it is a small edit away from being strong.
A local path passes the evidence check
Test the prefixes explicitly. http://localhost, file:// and a bare leading / are all unopenable by a reader, and none of them is caught by simply checking for http.
Include the loopback IPs as well. http://127.0.0.1:8080 and https://[::1] are exactly as unopenable as the name they stand for, and people write both — a checker that knows only the word localhost passes the numeric form straight through.
test_every_missing_property_is_named fails
"Built a robust system." should produce exactly four reasons: vague wording, no measurement, unattributed, and no evidence link. If you get three, check that the measurement branch runs independently of the vague-word branch rather than inside an elif.
attribution returns "mine" for "We improved my model"
That is correct — both singular and team wording are present, so the answer should be mixed. If yours returns mine, you are returning early on the first match instead of testing both before deciding.
The measurement pattern matches a version number
v1.2 and Python 3.14 are numbers and are not results. This is a real limitation of a regular-expression approach and the reference implementation shares it: the unit requirement suppresses most cases, and a claim like "upgraded to 3.14" can still slip through. Worth knowing rather than worth over-engineering around.
NotImplementedError on nearly every test
Expected. The starter stubs five functions — see expected-output/starter-run.txt, which also names the one test that passes without them.
Security notes
Security notes — Day 363
What this lab does and does not touch
- No network. Evidence links are judged from the string rather than fetched. No test opens a socket.
- No credentials. No API key, token or account.
- No writes outside this directory. Only
__pycache__/,.pytest_cache/and, if you create it,.venv/. - No
sudo, no global installs. - The example claims are fictional, and the URLs use
example.com, a domain reserved by RFC 2606.
A portfolio is a permanent public disclosure
This is the security point of the day, and it is easy to skip past because a portfolio does not feel like infrastructure.
Once published, it is copied, cached, indexed and forked. Deleting it later removes your copy and none of the others. Four things leak this way routinely:
- Secrets in repository history. Making a private repository public exposes every commit, not just the current tree. A key removed six months ago is still there. Run
gitleaksortrufflehogover the history before flipping visibility, not after. - Screenshots. They capture whatever was on screen — a token in a shell prompt, an internal hostname in a browser tab, another window. Crop deliberately and look at what remains.
- Demo recordings. Same problem, over a longer window. A terminal recording that captures text rather than pixels at least lets you read it before publishing.
- Real data in a demo corpus. Publishing the demo publishes the corpus. Use fixtures you own.
Work you may not be free to publish
Work done for an employer is frequently not yours to release, and the constraint is contractual rather than technical. Two routes that keep you on the right side of it:
- Describe without publishing. A claim can be specific, measured and attributed without the artefact being public. The evidence check will grade it weak, and that is the correct outcome — the reader genuinely cannot verify it, and pretending otherwise helps nobody.
- Rebuild it on your own time, on public data, as a comparable thing you can publish.
What not to do is publish it anyway and hope. That decision is discoverable, permanent, and says something about you that no portfolio recovers from.
A hosted demo is an exposed endpoint
If you host a live demo, it is a public service with everything that implies: it can be scraped, abused and made expensive. If it calls a paid API, put a hard cap on it — Day 327's exercise — or ship a recording instead, which costs nothing and cannot be abused.
Reversing everything this lab did
rm -rf __pycache__ .pytest_cache .venv
Nothing else was created, and nothing outside this directory was modified.