Capstone › Capstone Build II: Ship It › Day 358
Hands-on lab — Day 358: Frontend and User Experience
- ← Back to the Day 358 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/capstone/day-358-frontend-and-user-experience/
Commands
Setup
pip install -r requirements/requirements.txt Run
python examples/stream_ux_demo.py Test
bash tests/run_tests.sh File tree
examples/stream_ux_demo.py examples/stream_ux.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/stream_ux.py tests/run_tests.sh tests/test_stream_ux.py troubleshooting.md
Lab README
Lab — Day 358: Frontend and User Experience
Lesson
- Lesson title: Frontend and User Experience
- Day number: 358 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-358-frontend-and-user-experience
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-358-frontend-and-user-experiencewhen the site is running.
Purpose
Implement the state machine behind a streaming AI interface, then measure what each design decision costs the person waiting.
Everything runs offline on a logical clock — one tick per observable moment — so the measurements are exact and can be asserted on, which a wall-clock version could not.
Learning objectives
- Model a response as five observable states and enforce that terminal states are terminal.
- Measure time to first token and perceived wait, and distinguish both from total duration.
- Keep partial output when a stream breaks.
- Treat cancellation as its own ending rather than an error.
- Show that text only grows during streaming and every frame differs from the last.
Prerequisites
- Day 357, "Milestone Review and Course Correction" — this is Week 52, and you are shipping the capstone you audited.
- Comfortable with Python generators, dataclasses and enums.
Supported operating systems
macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.
Hardware requirements
Any machine running Python 3.10 or newer. No GPU. The suite completes in hundredths of a second.
Required software
- Python 3.10 or newer.
pytest7 or newer.
Free and open-source options
Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. No account, no API key, no network — the token stream is simulated so the state machine is testable. The lesson discusses server-sent events, HTMX, the Vercel AI SDK, Streamlit and Gradio as the real options.
Installation
python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt
Removing .venv fully reverses the installation.
File structure
starter/stream_ux.py your work: render_stream, perceived_wait
examples/stream_ux.py reference implementation
examples/stream_ux_demo.py the same answer rendered five ways
tests/test_stream_ux.py grouped by UX property
tests/run_tests.sh suite entry point
expected-output/ real captured output and measured values
requirements/ pinned dependency
How to run
python3 examples/stream_ux_demo.py # five renderings of one answer
bash tests/run_tests.sh # run the suite
To work on the exercise, edit starter/stream_ux.py, then copy it over the reference:
cp starter/stream_ux.py examples/stream_ux.py
bash tests/run_tests.sh
Keep a copy of the reference first if you want to compare.
What the commands do
python3 examples/stream_ux_demo.py renders the same eight-token answer five ways — healthy streaming with every frame shown, a blocking equivalent, a stream that fails midway with the partial kept, the same failure with it discarded, and a user cancellation — reporting the ending, total ticks, time to first token, characters retained and perceived wait for each.
bash tests/run_tests.sh runs pytest over sixteen tests grouped by property: the state machine, perceived latency, incremental output, partial output, cancellation and the token source.
Expected output
--- streaming, healthy ---
t=0 idle ''
t=1 waiting '' awaiting first token
t=4 streaming 'The '
t=12 done 'The refund window is thirty day...'
done after 12 ticks, ttft=4, 47 chars perceived_wait=4
--- blocking (no partial output) ---
done after 20 ticks, ttft=12, 47 chars perceived_wait=12
--- stream fails midway, partial kept ---
error after 8 ticks, ttft=4, 21 chars perceived_wait=4
--- stream fails midway, partial discarded ---
error after 8 ticks, ttft=4, 0 chars perceived_wait=4
--- cancelled by user ---
cancelled after 7 ticks, ttft=4, 18 chars perceived_wait=4
(The healthy run prints every frame; only the first and last are shown here.) expected-output/FIELDS.md explains each field.
Validation steps
bash tests/run_tests.shreports16 passed.- Compare the first two summaries: identical content, perceived wait of 4 against 12.
- Compare the two failure runs: identical timing, and one keeps 21 characters while the other keeps none.
- Confirm no frame follows a terminal state. If one does, a cancelled response can still gain text.
Tests
Sixteen tests in six groups:
- the state machine — a healthy run reaches
done, states occur in a legal order, and no frame follows a terminal state. - perceived latency — streaming beats blocking on perceived wait,
ttftignores the waiting frames, perceived wait is not total duration, and a run that never emits has no first token. - incremental output — text only grows and every frame differs; waiting frames show nothing.
- partial output — a failure keeps what arrived by default, discarding loses the user's reading, and a failure before any token has nothing to keep.
- cancellation — stops early, keeps the partial text, and is not an error.
- the source — raises where configured, completes otherwise.
Cleanup
rm -rf __pycache__ .pytest_cache
rm -rf .venv
Both are safe to remove and fully reversible.
Troubleshooting
See troubleshooting.md.
Security notes
See security.md. In short: no network, no credentials — and the lesson covers why incremental rendering makes output escaping easier to get wrong.
Extension exercises
- Reconnection. Add a resume attempt after a failure, keeping text across both attempts, and test the failed-resume case as well as the successful one.
- Structured streaming. Stream JSON and render only complete top-level fields, then measure how much later the first useful render happens than with prose.
- A stall detector. Add a state for "streaming but nothing has arrived recently", which is a different problem from waiting and usually needs a different message.
Navigation
- Lesson
- Previous: Day 357 — Milestone Review and Course Correction
- Next: Day 359 — Deploying Your Capstone
Expected output
FIELDS.md
# Reading the transcript
## A frame line
t=5 streaming 'The refund '
| Part | Meaning |
| --- | --- |
| `t=` | Logical tick. One per observable moment, so the numbers are exact and assertable. |
| state | `idle`, `waiting`, `streaming`, `done`, `error` or `cancelled`. |
| text | Exactly what the interface would be showing at that moment. Truncated in the display, not in the data. |
| note | Present on waiting and on terminal frames — why it is waiting, or how it ended. |
## A summary line
done after 12 ticks, ttft=4, 47 chars perceived_wait=4
| Field | Meaning |
| --- | --- |
| ending | Which of the three terminal states was reached. |
| ticks | Total duration. **Not** what the user experiences. |
| `ttft` | Time to first token — the tick at which anything was first readable. `never` if nothing ever appeared. |
| chars | Length of the text the user is left with. Zero after a discarded partial. |
| `perceived_wait` | Time spent with nothing to read. Equals `ttft` when there is one, and the whole run when there is not. |
## The comparison to make
streaming done after 12 ticks, ttft=4, perceived_wait=4
blocking done after 20 ticks, ttft=12, perceived_wait=12
Same content. The streaming run is better on both numbers here, but the one that matters is the second: even if total duration were identical, the user would wait a third as long with nothing to read.
partial kept error after 8 ticks, 21 chars
partial discarded error after 8 ticks, 0 chars
Identical failure, identical timing. One version also takes away what the user had already read.
examples-run.txt
--- streaming, healthy ---
t=0 idle ''
t=1 waiting '' awaiting first token
t=2 waiting '' awaiting first token
t=3 waiting '' awaiting first token
t=4 streaming 'The '
t=5 streaming 'The refund '
t=6 streaming 'The refund window '
t=7 streaming 'The refund window is '
t=8 streaming 'The refund window is thirty '
t=9 streaming 'The refund window is thirty days '
t=10 streaming 'The refund window is thirty day...'
t=11 streaming 'The refund window is thirty day...'
t=12 done 'The refund window is thirty day...'
done after 12 ticks, ttft=4, 47 chars perceived_wait=4
--- blocking (no partial output) ---
done after 20 ticks, ttft=12, 47 chars perceived_wait=12
--- stream fails midway, partial kept ---
error after 8 ticks, ttft=4, 21 chars perceived_wait=4
--- stream fails midway, partial discarded ---
error after 8 ticks, ttft=4, 0 chars perceived_wait=4
--- cancelled by user ---
cancelled after 7 ticks, ttft=4, 18 chars perceived_wait=4
measured-values.txt
Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.
python3 examples/stream_ux_demo.py -> 20 lines, exit 0
bash tests/run_tests.sh -> 16 passed in 0.02s
Time is a LOGICAL clock -- one tick per observable moment -- so these numbers
are exact rather than "about right" on a busy laptop. A wall-clock version of
this lab would produce a different answer every run and could not be asserted
on.
The five runs, all delivering the same 47-character answer:
run ending total ttft chars felt
streaming, healthy done 12 4 47 4
blocking (no partial output) done 20 12 47 12
stream fails midway, partial kept error 8 4 21 4
stream fails midway, discarded error 8 4 0 4
cancelled by user cancelled 7 4 18 4
Two comparisons carry the lesson.
Rows 1 and 2 deliver identical content and the perceived wait is 4 against 12.
Streaming is not less work; it is an earlier first moment.
Rows 3 and 4 fail at the same instant, in the same number of ticks. The only
difference is that one of them also threw away 21 characters the user had
already read.
run-output.txt
--- streaming, healthy ---
t=0 idle ''
t=1 waiting '' awaiting first token
t=2 waiting '' awaiting first token
t=3 waiting '' awaiting first token
t=4 streaming 'The '
t=5 streaming 'The refund '
t=6 streaming 'The refund window '
t=7 streaming 'The refund window is '
t=8 streaming 'The refund window is thirty '
t=9 streaming 'The refund window is thirty days '
t=10 streaming 'The refund window is thirty day...'
t=11 streaming 'The refund window is thirty day...'
t=12 done 'The refund window is thirty day...'
done after 12 ticks, ttft=4, 47 chars perceived_wait=4
--- blocking (no partial output) ---
done after 20 ticks, ttft=12, 47 chars perceived_wait=12
--- stream fails midway, partial kept ---
error after 8 ticks, ttft=4, 21 chars perceived_wait=4
--- stream fails midway, partial discarded ---
error after 8 ticks, ttft=4, 0 chars perceived_wait=4
--- cancelled by user ---
cancelled after 7 ticks, ttft=4, 18 chars perceived_wait=4
starter-run.txt
An untouched starter fails most of the suite:
14 failed, 2 passed in 0.05s
The two that pass exercise token_source, which is not stubbed:
test_token_source_raises_at_the_configured_point
test_token_source_completes_when_it_does_not_fail
Everything else depends on render_stream or perceived_wait.
test-output.txt
................ [100%]
16 passed in 0.01s
test-run.txt
................ [100%]
16 passed in 0.01s
Source files
examples/stream_ux_demo.py (1153 bytes)
#!/usr/bin/env python3
"""Four runs of the same answer, showing what each UX decision costs."""
from __future__ import annotations
import os
import sys
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
from stream_ux import perceived_wait, render_stream
TOKENS = ["The ", "refund ", "window ", "is ", "thirty ", "days ", "from ", "purchase."]
def show(label: str, transcript, *, frames: bool = False) -> None:
print(f"--- {label} ---")
if frames:
for frame in transcript.frames:
print(frame.line())
print(f"{transcript.summary()} perceived_wait={perceived_wait(transcript)}")
def main() -> int:
show("streaming, healthy", render_stream(TOKENS), frames=True)
show("blocking (no partial output)", render_stream(TOKENS, latency_ticks=11))
show("stream fails midway, partial kept", render_stream(TOKENS, fail_at=4))
show(
"stream fails midway, partial discarded",
render_stream(TOKENS, fail_at=4, keep_partial_on_error=False),
)
show("cancelled by user", render_stream(TOKENS, cancel_at=3))
return 0
if __name__ == "__main__":
raise SystemExit(main())
examples/stream_ux.py (5554 bytes)
"""The state machine behind a streaming AI interface.
Offline, standard-library only, and deterministic -- time is a logical clock, so
the measurements are exact rather than "about right" on a busy laptop.
An AI interface is not a form that submits and returns. It is a long-running,
interruptible, partially-failing operation shown to someone who is waiting, and
almost everything that makes it feel good or bad lives in the state machine
rather than in the styling:
IDLE -> WAITING -> STREAMING -> DONE
\\-> ERROR
\\-> CANCELLED
Three properties decide whether it feels responsive:
time to first token what the user experiences as "did it hear me"
incremental delivery partial text is visible while more arrives
cancellability a wrong answer can be stopped without waiting
The last one is the one prototypes skip, and the one users reach for most.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from typing import Iterator
class State(str, Enum):
IDLE = "idle"
WAITING = "waiting"
STREAMING = "streaming"
DONE = "done"
ERROR = "error"
CANCELLED = "cancelled"
class StreamFailed(RuntimeError):
"""The upstream stream broke partway through."""
@dataclass
class Frame:
"""One observable moment, as a UI would render it."""
tick: int
state: State
text: str
note: str = ""
def line(self) -> str:
shown = self.text if len(self.text) <= 34 else self.text[:31] + "..."
base = f"t={self.tick:<3} {self.state.value:<9} {shown!r}"
return f"{base} {self.note}" if self.note else base
@dataclass
class Transcript:
frames: list[Frame] = field(default_factory=list)
@property
def final(self) -> Frame:
return self.frames[-1]
def time_to_first_token(self) -> int | None:
"""Ticks until the user saw ANY output. None if they never did."""
for frame in self.frames:
if frame.state is State.STREAMING and frame.text:
return frame.tick
return None
def states(self) -> list[State]:
return [f.state for f in self.frames]
def summary(self) -> str:
ttft = self.time_to_first_token()
return (
f"{self.final.state.value} after {self.final.tick} ticks, "
f"ttft={ttft if ttft is not None else 'never'}, "
f"{len(self.final.text)} chars"
)
def token_source(tokens: list[str], *, fail_at: int | None = None) -> Iterator[str]:
"""Yield tokens, optionally breaking partway through.
A stream that fails after emitting real content is the interesting case:
the user has already read something, so discarding it is a worse outcome
than keeping it and saying the answer is incomplete.
"""
for index, token in enumerate(tokens):
if fail_at is not None and index == fail_at:
raise StreamFailed(f"upstream closed after {index} tokens")
yield token
def render_stream(
tokens: list[str],
*,
latency_ticks: int = 3,
fail_at: int | None = None,
cancel_at: int | None = None,
keep_partial_on_error: bool = True,
) -> Transcript:
"""Drive the state machine and record every frame a UI would show.
`latency_ticks` is the wait before the first token -- the part the user
experiences as the system not having heard them.
"""
transcript = Transcript()
tick = 0
text = ""
transcript.frames.append(Frame(tick, State.IDLE, ""))
# Waiting: nothing to render yet. A skeleton or a typing indicator goes
# here, because an empty screen reads as a broken screen.
for _ in range(latency_ticks):
tick += 1
transcript.frames.append(Frame(tick, State.WAITING, "", "awaiting first token"))
stream = token_source(tokens, fail_at=fail_at)
emitted = 0
while True:
try:
token = next(stream)
except StopIteration:
tick += 1
transcript.frames.append(Frame(tick, State.DONE, text))
return transcript
except StreamFailed as exc:
tick += 1
# Keeping what arrived is almost always right: the user has already
# read it, and replacing it with an error box discards work they
# were relying on.
kept = text if keep_partial_on_error else ""
transcript.frames.append(Frame(tick, State.ERROR, kept, str(exc)))
return transcript
tick += 1
text += token
emitted += 1
transcript.frames.append(Frame(tick, State.STREAMING, text))
if cancel_at is not None and emitted >= cancel_at:
tick += 1
# Cancellation keeps the partial text too. The user asked it to
# stop, not to undo.
transcript.frames.append(
Frame(tick, State.CANCELLED, text, "stopped by user")
)
return transcript
def perceived_wait(transcript: Transcript) -> int:
"""Ticks the user spent with nothing to read.
This is the number that correlates with how slow an interface feels, and it
is NOT the total duration. A response that streams for forty ticks after a
three-tick wait feels faster than one that appears complete after ten.
"""
ttft = transcript.time_to_first_token()
return transcript.final.tick if ttft is None else ttft
def is_terminal(state: State) -> bool:
return state in (State.DONE, State.ERROR, State.CANCELLED)
metadata.yml (831 bytes)
lesson_id: D358
day: 358
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/stream_ux_demo.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3
examples/stream_ux_demo.py reproduced the documented five runs exactly; bash tests/run_tests.sh -> 16
passed in 0.02s. The untouched starter fails 14 of 16; the 2 that pass exercise token_source, which is
not stubbed. requires_network is false: the token stream is simulated and time is a logical clock, so
the measurements are exact rather than machine-dependent.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/stream_ux.py (5169 bytes)
"""The state machine behind a streaming AI interface.
Offline, standard-library only, and deterministic -- time is a logical clock, so
the measurements are exact rather than "about right" on a busy laptop.
An AI interface is not a form that submits and returns. It is a long-running,
interruptible, partially-failing operation shown to someone who is waiting, and
almost everything that makes it feel good or bad lives in the state machine
rather than in the styling:
IDLE -> WAITING -> STREAMING -> DONE
\\-> ERROR
\\-> CANCELLED
Three properties decide whether it feels responsive:
time to first token what the user experiences as "did it hear me"
incremental delivery partial text is visible while more arrives
cancellability a wrong answer can be stopped without waiting
The last one is the one prototypes skip, and the one users reach for most.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from typing import Iterator
class State(str, Enum):
IDLE = "idle"
WAITING = "waiting"
STREAMING = "streaming"
DONE = "done"
ERROR = "error"
CANCELLED = "cancelled"
class StreamFailed(RuntimeError):
"""The upstream stream broke partway through."""
@dataclass
class Frame:
"""One observable moment, as a UI would render it."""
tick: int
state: State
text: str
note: str = ""
def line(self) -> str:
shown = self.text if len(self.text) <= 34 else self.text[:31] + "..."
base = f"t={self.tick:<3} {self.state.value:<9} {shown!r}"
return f"{base} {self.note}" if self.note else base
@dataclass
class Transcript:
frames: list[Frame] = field(default_factory=list)
@property
def final(self) -> Frame:
return self.frames[-1]
def time_to_first_token(self) -> int | None:
"""Ticks until the user saw ANY output. None if they never did."""
for frame in self.frames:
if frame.state is State.STREAMING and frame.text:
return frame.tick
return None
def states(self) -> list[State]:
return [f.state for f in self.frames]
def summary(self) -> str:
ttft = self.time_to_first_token()
return (
f"{self.final.state.value} after {self.final.tick} ticks, "
f"ttft={ttft if ttft is not None else 'never'}, "
f"{len(self.final.text)} chars"
)
def token_source(tokens: list[str], *, fail_at: int | None = None) -> Iterator[str]:
"""Yield tokens, optionally breaking partway through.
A stream that fails after emitting real content is the interesting case:
the user has already read something, so discarding it is a worse outcome
than keeping it and saying the answer is incomplete.
"""
for index, token in enumerate(tokens):
if fail_at is not None and index == fail_at:
raise StreamFailed(f"upstream closed after {index} tokens")
yield token
def render_stream(
tokens: list[str],
*,
latency_ticks: int = 3,
fail_at: int | None = None,
cancel_at: int | None = None,
keep_partial_on_error: bool = True,
) -> Transcript:
"""Drive the state machine and record every frame a UI would show.
`latency_ticks` is the wait before the first token -- the part the user
experiences as the system not having heard them.
"""
# TASK 1: drive the state machine, appending one Frame per observable
# moment.
# - start at tick 0 with State.IDLE and empty text
# - emit `latency_ticks` WAITING frames with empty text; an unchanged
# screen is indistinguishable from a broken one
# - for each token: advance the tick, APPEND to the accumulated text, and
# emit a STREAMING frame (text only ever grows, and every frame differs)
# - StopIteration -> a DONE frame
# - StreamFailed -> an ERROR frame keeping the text so far, unless
# keep_partial_on_error is False. The user already read it.
# - after `cancel_at` tokens -> a CANCELLED frame keeping the text, with
# note "stopped by user". Not an error: they stopped it deliberately.
# - a terminal frame ENDS the run. Return immediately, or a late token
# appends to a response the user already stopped.
raise NotImplementedError("implement render_stream")
def perceived_wait(transcript: Transcript) -> int:
"""Ticks the user spent with nothing to read.
This is the number that correlates with how slow an interface feels, and it
is NOT the total duration. A response that streams for forty ticks after a
three-tick wait feels faster than one that appears complete after ten.
"""
# TASK 2: ticks the user spent with nothing to read.
# That is the time to first token when there is one, and the WHOLE run when
# there is not. It is not the total duration, and confusing the two is the
# most common mistake in this area.
raise NotImplementedError("implement perceived_wait")
def is_terminal(state: State) -> bool:
return state in (State.DONE, State.ERROR, State.CANCELLED)
tests/run_tests.sh (202 bytes)
#!/usr/bin/env bash
# Run the day 358 streaming UX test suite.
# Offline, standard library plus pytest. No network, no API key.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_stream_ux.py (4666 bytes)
"""Grouped by UX property, so a failure names what the user would feel.
Run with: bash tests/run_tests.sh
"""
from __future__ import annotations
import os
import sys
import pytest
HERE = os.path.dirname(os.path.abspath(__file__))
LAB = os.path.dirname(HERE)
sys.path.insert(0, os.path.join(LAB, "examples"))
from stream_ux import ( # noqa: E402
State,
StreamFailed,
is_terminal,
perceived_wait,
render_stream,
token_source,
)
TOKENS = ["The ", "refund ", "window ", "is ", "thirty ", "days."]
# ------------------------------------------------------- the state machine
def test_healthy_run_reaches_done():
t = render_stream(TOKENS)
assert t.final.state is State.DONE
assert t.final.text == "".join(TOKENS)
def test_states_occur_in_a_legal_order():
states = render_stream(TOKENS).states()
assert states[0] is State.IDLE
assert State.WAITING in states
assert states.index(State.WAITING) < states.index(State.STREAMING)
assert is_terminal(states[-1])
def test_no_frames_follow_a_terminal_state():
for kwargs in ({}, {"fail_at": 3}, {"cancel_at": 2}):
t = render_stream(TOKENS, **kwargs)
terminal = [i for i, f in enumerate(t.frames) if is_terminal(f.state)]
assert terminal == [len(t.frames) - 1], "a terminal state must end the run"
# ------------------------------------------------------ perceived latency
def test_streaming_is_perceived_as_faster_than_blocking():
# Same content, same total work. The difference is entirely when the user
# first has something to read.
streaming = render_stream(TOKENS, latency_ticks=3)
blocking = render_stream(TOKENS, latency_ticks=9)
assert perceived_wait(streaming) < perceived_wait(blocking)
def test_time_to_first_token_ignores_the_waiting_frames():
t = render_stream(TOKENS, latency_ticks=5)
assert t.time_to_first_token() == 6 # 5 waiting frames, then the first token
def test_perceived_wait_is_not_total_duration():
t = render_stream(TOKENS, latency_ticks=2)
assert perceived_wait(t) < t.final.tick
def test_a_run_that_never_emits_has_no_first_token():
t = render_stream([], latency_ticks=2)
assert t.time_to_first_token() is None
# With nothing ever shown, the whole run is perceived wait.
assert perceived_wait(t) == t.final.tick
# ----------------------------------------------------- incremental output
def test_text_only_grows_during_streaming():
t = render_stream(TOKENS)
lengths = [len(f.text) for f in t.frames if f.state is State.STREAMING]
assert lengths == sorted(lengths)
assert len(set(lengths)) == len(lengths), "each frame should add something"
def test_waiting_frames_show_nothing():
t = render_stream(TOKENS, latency_ticks=4)
assert all(f.text == "" for f in t.frames if f.state is State.WAITING)
# --------------------------------------------------------- partial output
def test_a_failed_stream_keeps_what_arrived_by_default():
t = render_stream(TOKENS, fail_at=3)
assert t.final.state is State.ERROR
assert t.final.text == "The refund window "
assert "upstream closed" in t.final.note
def test_discarding_partial_output_loses_what_the_user_read():
kept = render_stream(TOKENS, fail_at=3)
discarded = render_stream(TOKENS, fail_at=3, keep_partial_on_error=False)
assert discarded.final.text == ""
assert len(kept.final.text) > 0
# Both failed at the same point; only one wasted the user's reading.
assert kept.final.tick == discarded.final.tick
def test_failure_before_any_token_leaves_nothing_to_keep():
t = render_stream(TOKENS, fail_at=0)
assert t.final.state is State.ERROR
assert t.final.text == ""
assert t.time_to_first_token() is None
# --------------------------------------------------------- cancellation
def test_cancellation_stops_early_and_keeps_the_partial_text():
t = render_stream(TOKENS, cancel_at=2)
assert t.final.state is State.CANCELLED
assert t.final.text == "The refund "
assert t.final.tick < render_stream(TOKENS).final.tick
def test_cancellation_is_not_an_error():
t = render_stream(TOKENS, cancel_at=2)
assert t.final.state is not State.ERROR
assert t.final.note == "stopped by user"
# --------------------------------------------------------- the source
def test_token_source_raises_at_the_configured_point():
gen = token_source(TOKENS, fail_at=2)
assert next(gen) == "The "
assert next(gen) == "refund "
with pytest.raises(StreamFailed):
next(gen)
def test_token_source_completes_when_it_does_not_fail():
assert list(token_source(TOKENS)) == TOKENS
Troubleshooting
Troubleshooting — Day 358
perceived_wait equals the total duration on every run
You are returning transcript.final.tick unconditionally. It should be the time to first token whenever there is one, and only fall back to the final tick when nothing was ever shown. Those are different quantities, and conflating them makes streaming look no better than blocking.
The blocking run does not look worse than the streaming one
You are comparing total duration. Compare perceived_wait, or time_to_first_token. In the reference run both deliver 47 characters and the difference is 4 ticks against 12.
test_no_frames_follow_a_terminal_state fails
Your loop appends the terminal frame and keeps going. Return immediately after appending done, error or cancelled. This is the bug that lets a late token append to a response the user already stopped — a stop button that does not stop.
The error case loses text that should be kept
You are clearing the accumulated text when the stream raises. Keep it unless keep_partial_on_error is False. The user has already read it; the failure does not un-read it.
test_text_only_grows_during_streaming fails on duplicate lengths
Every streaming frame must add something. If two consecutive frames have the same length you are re-emitting the buffer rather than appending a token — usually a frame appended before the token is added rather than after.
time_to_first_token returns a waiting tick
It must find the first frame that is STREAMING and has non-empty text. A waiting frame has empty text by construction, so checking the state alone is not enough if you ever emit an empty streaming frame.
Cancellation is reported as an error
cancel_at should produce State.CANCELLED with the note "stopped by user", keeping the partial text. Reporting it as an error tells the user something broke when they were the one who stopped it.
A run with no tokens behaves strangely
render_stream([]) is a legitimate case: the stream completes without ever emitting. time_to_first_token should return None, and perceived_wait should then be the whole run — the user waited for the entire duration and saw nothing.
NotImplementedError on most tests
Expected. The starter stubs render_stream and perceived_wait — see expected-output/starter-run.txt, which also names the two tests that pass without them.
Security notes
Security notes — Day 358
What this lab does and does not touch
- No network. The token stream is a generator over a fixed list. No test opens a socket.
- No credentials. No API key, token or account.
- No writes outside this directory. Only
__pycache__/,.pytest_cache/and, if you create it,.venv/. - No
sudo, no global installs. - No real clock. Time is a logical tick counter, so the lab cannot be affected by, or affect, system timing.
Why streaming makes escaping harder, and what to do about it
This lab renders plain strings, so nothing here is exploitable. A real interface renders into a document, and incremental rendering changes the threat model in a specific way:
A sanitiser written to process a complete document sees a partial one. A tag that arrives split across two tokens — <scr then ipt> — is not a tag when either half is examined alone. Naive "sanitise the buffer each time" logic can therefore pass content that would have been caught in a single pass over the finished text.
The rule is: escape on every append, and escape the token you are adding rather than re-processing the accumulated buffer. Treat model output as untrusted text throughout, never as markup, and never as HTML you assemble by concatenation.
Whole-response checks conflict with streaming
Streaming means the first tokens are visible before the last ones exist. Any check that needs the complete response — a moderation pass, a policy filter, a schema validation — cannot run before the user has already read part of the answer.
This is a genuine trade-off rather than a bug, and it has three honest resolutions:
- Block for content that must be checked. Accept the perceived-latency cost where it is warranted.
- Check incrementally where the check permits it, accepting that a violation may be briefly visible before it is caught.
- Stream to a holding area the user does not see until a check passes, which is blocking with extra steps and sometimes the clearest way to describe it.
Choose deliberately. Streaming by default, then discovering the moderation pass has nowhere to run, is the failure this note exists to prevent.
Partial and cancelled responses are still content
A partial answer left on screen after an error, and a cancelled response the user stopped reading, were both generated. They may be in your logs, your traces and your cost records. They carry the same retention, privacy and logging obligations as a complete answer — a point that is easy to miss because neither "finished".
Reversing everything this lab did
rm -rf __pycache__ .pytest_cache .venv
Nothing else was created, and nothing outside this directory was modified.