Capstone › Capstone Build II: Ship It › Day 358

Hands-on lab — Day 358: Frontend and User Experience

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/stream_ux_demo.py

Test

bash tests/run_tests.sh

File tree

examples/stream_ux_demo.py
examples/stream_ux.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/stream_ux.py
tests/run_tests.sh
tests/test_stream_ux.py
troubleshooting.md

Lab README

Lab — Day 358: Frontend and User Experience

Lesson

Purpose

Implement the state machine behind a streaming AI interface, then measure what each design decision costs the person waiting.

Everything runs offline on a logical clock — one tick per observable moment — so the measurements are exact and can be asserted on, which a wall-clock version could not.

Learning objectives

  • Model a response as five observable states and enforce that terminal states are terminal.
  • Measure time to first token and perceived wait, and distinguish both from total duration.
  • Keep partial output when a stream breaks.
  • Treat cancellation as its own ending rather than an error.
  • Show that text only grows during streaming and every frame differs from the last.

Prerequisites

  • Day 357, "Milestone Review and Course Correction" — this is Week 52, and you are shipping the capstone you audited.
  • Comfortable with Python generators, dataclasses and enums.

Supported operating systems

macOS, Linux and Windows (via WSL or PowerShell). Nothing is platform-specific.

Hardware requirements

Any machine running Python 3.10 or newer. No GPU. The suite completes in hundredths of a second.

Required software

  • Python 3.10 or newer.
  • pytest 7 or newer.

Free and open-source options

Every dependency is free and open source: Python is PSF-licensed, pytest is MIT-licensed. No account, no API key, no network — the token stream is simulated so the state machine is testable. The lesson discusses server-sent events, HTMX, the Vercel AI SDK, Streamlit and Gradio as the real options.

Installation

python3 -m venv .venv
.venv/bin/pip install -r requirements/requirements.txt

Removing .venv fully reverses the installation.

File structure

starter/stream_ux.py         your work: render_stream, perceived_wait
examples/stream_ux.py        reference implementation
examples/stream_ux_demo.py   the same answer rendered five ways
tests/test_stream_ux.py      grouped by UX property
tests/run_tests.sh           suite entry point
expected-output/             real captured output and measured values
requirements/                pinned dependency

How to run

python3 examples/stream_ux_demo.py   # five renderings of one answer
bash tests/run_tests.sh              # run the suite

To work on the exercise, edit starter/stream_ux.py, then copy it over the reference:

cp starter/stream_ux.py examples/stream_ux.py
bash tests/run_tests.sh

Keep a copy of the reference first if you want to compare.

What the commands do

python3 examples/stream_ux_demo.py renders the same eight-token answer five ways — healthy streaming with every frame shown, a blocking equivalent, a stream that fails midway with the partial kept, the same failure with it discarded, and a user cancellation — reporting the ending, total ticks, time to first token, characters retained and perceived wait for each.

bash tests/run_tests.sh runs pytest over sixteen tests grouped by property: the state machine, perceived latency, incremental output, partial output, cancellation and the token source.

Expected output

--- streaming, healthy ---
t=0   idle      ''
t=1   waiting   ''  awaiting first token
t=4   streaming 'The '
t=12  done      'The refund window is thirty day...'
done after 12 ticks, ttft=4, 47 chars  perceived_wait=4
--- blocking (no partial output) ---
done after 20 ticks, ttft=12, 47 chars  perceived_wait=12
--- stream fails midway, partial kept ---
error after 8 ticks, ttft=4, 21 chars  perceived_wait=4
--- stream fails midway, partial discarded ---
error after 8 ticks, ttft=4, 0 chars  perceived_wait=4
--- cancelled by user ---
cancelled after 7 ticks, ttft=4, 18 chars  perceived_wait=4

(The healthy run prints every frame; only the first and last are shown here.) expected-output/FIELDS.md explains each field.

Validation steps

  1. bash tests/run_tests.sh reports 16 passed.
  2. Compare the first two summaries: identical content, perceived wait of 4 against 12.
  3. Compare the two failure runs: identical timing, and one keeps 21 characters while the other keeps none.
  4. Confirm no frame follows a terminal state. If one does, a cancelled response can still gain text.

Tests

Sixteen tests in six groups:

  • the state machine — a healthy run reaches done, states occur in a legal order, and no frame follows a terminal state.
  • perceived latency — streaming beats blocking on perceived wait, ttft ignores the waiting frames, perceived wait is not total duration, and a run that never emits has no first token.
  • incremental output — text only grows and every frame differs; waiting frames show nothing.
  • partial output — a failure keeps what arrived by default, discarding loses the user's reading, and a failure before any token has nothing to keep.
  • cancellation — stops early, keeps the partial text, and is not an error.
  • the source — raises where configured, completes otherwise.

Cleanup

rm -rf __pycache__ .pytest_cache
rm -rf .venv

Both are safe to remove and fully reversible.

Troubleshooting

See troubleshooting.md.

Security notes

See security.md. In short: no network, no credentials — and the lesson covers why incremental rendering makes output escaping easier to get wrong.

Extension exercises

  1. Reconnection. Add a resume attempt after a failure, keeping text across both attempts, and test the failed-resume case as well as the successful one.
  2. Structured streaming. Stream JSON and render only complete top-level fields, then measure how much later the first useful render happens than with prose.
  3. A stall detector. Add a state for "streaming but nothing has arrived recently", which is a different problem from waiting and usually needs a different message.
  • Lesson
  • Previous: Day 357 — Milestone Review and Course Correction
  • Next: Day 359 — Deploying Your Capstone

Expected output

FIELDS.md

# Reading the transcript

## A frame line

    t=5   streaming 'The refund '

| Part | Meaning |
| --- | --- |
| `t=` | Logical tick. One per observable moment, so the numbers are exact and assertable. |
| state | `idle`, `waiting`, `streaming`, `done`, `error` or `cancelled`. |
| text | Exactly what the interface would be showing at that moment. Truncated in the display, not in the data. |
| note | Present on waiting and on terminal frames — why it is waiting, or how it ended. |

## A summary line

    done after 12 ticks, ttft=4, 47 chars  perceived_wait=4

| Field | Meaning |
| --- | --- |
| ending | Which of the three terminal states was reached. |
| ticks | Total duration. **Not** what the user experiences. |
| `ttft` | Time to first token — the tick at which anything was first readable. `never` if nothing ever appeared. |
| chars | Length of the text the user is left with. Zero after a discarded partial. |
| `perceived_wait` | Time spent with nothing to read. Equals `ttft` when there is one, and the whole run when there is not. |

## The comparison to make

    streaming   done after 12 ticks, ttft=4,  perceived_wait=4
    blocking    done after 20 ticks, ttft=12, perceived_wait=12

Same content. The streaming run is better on both numbers here, but the one that matters is the second: even if total duration were identical, the user would wait a third as long with nothing to read.

    partial kept       error after 8 ticks, 21 chars
    partial discarded  error after 8 ticks,  0 chars

Identical failure, identical timing. One version also takes away what the user had already read.

examples-run.txt

--- streaming, healthy ---
t=0   idle      ''
t=1   waiting   ''  awaiting first token
t=2   waiting   ''  awaiting first token
t=3   waiting   ''  awaiting first token
t=4   streaming 'The '
t=5   streaming 'The refund '
t=6   streaming 'The refund window '
t=7   streaming 'The refund window is '
t=8   streaming 'The refund window is thirty '
t=9   streaming 'The refund window is thirty days '
t=10  streaming 'The refund window is thirty day...'
t=11  streaming 'The refund window is thirty day...'
t=12  done      'The refund window is thirty day...'
done after 12 ticks, ttft=4, 47 chars  perceived_wait=4
--- blocking (no partial output) ---
done after 20 ticks, ttft=12, 47 chars  perceived_wait=12
--- stream fails midway, partial kept ---
error after 8 ticks, ttft=4, 21 chars  perceived_wait=4
--- stream fails midway, partial discarded ---
error after 8 ticks, ttft=4, 0 chars  perceived_wait=4
--- cancelled by user ---
cancelled after 7 ticks, ttft=4, 18 chars  perceived_wait=4

measured-values.txt

Measured on macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1.

  python3 examples/stream_ux_demo.py -> 20 lines, exit 0
  bash tests/run_tests.sh            -> 16 passed in 0.02s

Time is a LOGICAL clock -- one tick per observable moment -- so these numbers
are exact rather than "about right" on a busy laptop. A wall-clock version of
this lab would produce a different answer every run and could not be asserted
on.

The five runs, all delivering the same 47-character answer:

  run                                 ending     total  ttft  chars  felt
  streaming, healthy                  done        12     4     47     4
  blocking (no partial output)        done        20    12     47    12
  stream fails midway, partial kept   error        8     4     21     4
  stream fails midway, discarded      error        8     4      0     4
  cancelled by user                   cancelled    7     4     18     4

Two comparisons carry the lesson.

Rows 1 and 2 deliver identical content and the perceived wait is 4 against 12.
Streaming is not less work; it is an earlier first moment.

Rows 3 and 4 fail at the same instant, in the same number of ticks. The only
difference is that one of them also threw away 21 characters the user had
already read.

run-output.txt

--- streaming, healthy ---
t=0   idle      ''
t=1   waiting   ''  awaiting first token
t=2   waiting   ''  awaiting first token
t=3   waiting   ''  awaiting first token
t=4   streaming 'The '
t=5   streaming 'The refund '
t=6   streaming 'The refund window '
t=7   streaming 'The refund window is '
t=8   streaming 'The refund window is thirty '
t=9   streaming 'The refund window is thirty days '
t=10  streaming 'The refund window is thirty day...'
t=11  streaming 'The refund window is thirty day...'
t=12  done      'The refund window is thirty day...'
done after 12 ticks, ttft=4, 47 chars  perceived_wait=4
--- blocking (no partial output) ---
done after 20 ticks, ttft=12, 47 chars  perceived_wait=12
--- stream fails midway, partial kept ---
error after 8 ticks, ttft=4, 21 chars  perceived_wait=4
--- stream fails midway, partial discarded ---
error after 8 ticks, ttft=4, 0 chars  perceived_wait=4
--- cancelled by user ---
cancelled after 7 ticks, ttft=4, 18 chars  perceived_wait=4

starter-run.txt

An untouched starter fails most of the suite:

    14 failed, 2 passed in 0.05s

The two that pass exercise token_source, which is not stubbed:

    test_token_source_raises_at_the_configured_point
    test_token_source_completes_when_it_does_not_fail

Everything else depends on render_stream or perceived_wait.

test-output.txt

................                                                         [100%]
16 passed in 0.01s

test-run.txt

................                                                         [100%]
16 passed in 0.01s

Source files

examples/stream_ux_demo.py (1153 bytes)
#!/usr/bin/env python3
"""Four runs of the same answer, showing what each UX decision costs."""

from __future__ import annotations

import os
import sys

sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

from stream_ux import perceived_wait, render_stream

TOKENS = ["The ", "refund ", "window ", "is ", "thirty ", "days ", "from ", "purchase."]


def show(label: str, transcript, *, frames: bool = False) -> None:
    print(f"--- {label} ---")
    if frames:
        for frame in transcript.frames:
            print(frame.line())
    print(f"{transcript.summary()}  perceived_wait={perceived_wait(transcript)}")


def main() -> int:
    show("streaming, healthy", render_stream(TOKENS), frames=True)
    show("blocking (no partial output)", render_stream(TOKENS, latency_ticks=11))
    show("stream fails midway, partial kept", render_stream(TOKENS, fail_at=4))
    show(
        "stream fails midway, partial discarded",
        render_stream(TOKENS, fail_at=4, keep_partial_on_error=False),
    )
    show("cancelled by user", render_stream(TOKENS, cancel_at=3))
    return 0


if __name__ == "__main__":
    raise SystemExit(main())
examples/stream_ux.py (5554 bytes)
"""The state machine behind a streaming AI interface.

Offline, standard-library only, and deterministic -- time is a logical clock, so
the measurements are exact rather than "about right" on a busy laptop.

An AI interface is not a form that submits and returns. It is a long-running,
interruptible, partially-failing operation shown to someone who is waiting, and
almost everything that makes it feel good or bad lives in the state machine
rather than in the styling:

  IDLE -> WAITING -> STREAMING -> DONE
                  \\-> ERROR
                  \\-> CANCELLED

Three properties decide whether it feels responsive:

  time to first token   what the user experiences as "did it hear me"
  incremental delivery  partial text is visible while more arrives
  cancellability        a wrong answer can be stopped without waiting

The last one is the one prototypes skip, and the one users reach for most.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum
from typing import Iterator


class State(str, Enum):
    IDLE = "idle"
    WAITING = "waiting"
    STREAMING = "streaming"
    DONE = "done"
    ERROR = "error"
    CANCELLED = "cancelled"


class StreamFailed(RuntimeError):
    """The upstream stream broke partway through."""


@dataclass
class Frame:
    """One observable moment, as a UI would render it."""

    tick: int
    state: State
    text: str
    note: str = ""

    def line(self) -> str:
        shown = self.text if len(self.text) <= 34 else self.text[:31] + "..."
        base = f"t={self.tick:<3} {self.state.value:<9} {shown!r}"
        return f"{base}  {self.note}" if self.note else base


@dataclass
class Transcript:
    frames: list[Frame] = field(default_factory=list)

    @property
    def final(self) -> Frame:
        return self.frames[-1]

    def time_to_first_token(self) -> int | None:
        """Ticks until the user saw ANY output. None if they never did."""
        for frame in self.frames:
            if frame.state is State.STREAMING and frame.text:
                return frame.tick
        return None

    def states(self) -> list[State]:
        return [f.state for f in self.frames]

    def summary(self) -> str:
        ttft = self.time_to_first_token()
        return (
            f"{self.final.state.value} after {self.final.tick} ticks, "
            f"ttft={ttft if ttft is not None else 'never'}, "
            f"{len(self.final.text)} chars"
        )


def token_source(tokens: list[str], *, fail_at: int | None = None) -> Iterator[str]:
    """Yield tokens, optionally breaking partway through.

    A stream that fails after emitting real content is the interesting case:
    the user has already read something, so discarding it is a worse outcome
    than keeping it and saying the answer is incomplete.
    """
    for index, token in enumerate(tokens):
        if fail_at is not None and index == fail_at:
            raise StreamFailed(f"upstream closed after {index} tokens")
        yield token


def render_stream(
    tokens: list[str],
    *,
    latency_ticks: int = 3,
    fail_at: int | None = None,
    cancel_at: int | None = None,
    keep_partial_on_error: bool = True,
) -> Transcript:
    """Drive the state machine and record every frame a UI would show.

    `latency_ticks` is the wait before the first token -- the part the user
    experiences as the system not having heard them.
    """
    transcript = Transcript()
    tick = 0
    text = ""

    transcript.frames.append(Frame(tick, State.IDLE, ""))

    # Waiting: nothing to render yet. A skeleton or a typing indicator goes
    # here, because an empty screen reads as a broken screen.
    for _ in range(latency_ticks):
        tick += 1
        transcript.frames.append(Frame(tick, State.WAITING, "", "awaiting first token"))

    stream = token_source(tokens, fail_at=fail_at)
    emitted = 0
    while True:
        try:
            token = next(stream)
        except StopIteration:
            tick += 1
            transcript.frames.append(Frame(tick, State.DONE, text))
            return transcript
        except StreamFailed as exc:
            tick += 1
            # Keeping what arrived is almost always right: the user has already
            # read it, and replacing it with an error box discards work they
            # were relying on.
            kept = text if keep_partial_on_error else ""
            transcript.frames.append(Frame(tick, State.ERROR, kept, str(exc)))
            return transcript

        tick += 1
        text += token
        emitted += 1
        transcript.frames.append(Frame(tick, State.STREAMING, text))

        if cancel_at is not None and emitted >= cancel_at:
            tick += 1
            # Cancellation keeps the partial text too. The user asked it to
            # stop, not to undo.
            transcript.frames.append(
                Frame(tick, State.CANCELLED, text, "stopped by user")
            )
            return transcript


def perceived_wait(transcript: Transcript) -> int:
    """Ticks the user spent with nothing to read.

    This is the number that correlates with how slow an interface feels, and it
    is NOT the total duration. A response that streams for forty ticks after a
    three-tick wait feels faster than one that appears complete after ten.
    """
    ttft = transcript.time_to_first_token()
    return transcript.final.tick if ttft is None else ttft


def is_terminal(state: State) -> bool:
    return state in (State.DONE, State.ERROR, State.CANCELLED)
metadata.yml (831 bytes)
lesson_id: D358
day: 358
kind: python
languages:
  - python
setup_commands:
  - pip install -r requirements/requirements.txt
run_commands:
  - python examples/stream_ux_demo.py
test_commands:
  - bash tests/run_tests.sh
cleanup_commands:
  - rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-31'
executed_on: 'macOS 26.5.1 (Apple Silicon), Python 3.14.0, pytest 9.1.1 — python3
  examples/stream_ux_demo.py reproduced the documented five runs exactly; bash tests/run_tests.sh -> 16
  passed in 0.02s. The untouched starter fails 14 of 16; the 2 that pass exercise token_source, which is
  not stubbed. requires_network is false: the token stream is simulated and time is a logical clock, so
  the measurements are exact rather than machine-dependent.'
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/stream_ux.py (5169 bytes)
"""The state machine behind a streaming AI interface.

Offline, standard-library only, and deterministic -- time is a logical clock, so
the measurements are exact rather than "about right" on a busy laptop.

An AI interface is not a form that submits and returns. It is a long-running,
interruptible, partially-failing operation shown to someone who is waiting, and
almost everything that makes it feel good or bad lives in the state machine
rather than in the styling:

  IDLE -> WAITING -> STREAMING -> DONE
                  \\-> ERROR
                  \\-> CANCELLED

Three properties decide whether it feels responsive:

  time to first token   what the user experiences as "did it hear me"
  incremental delivery  partial text is visible while more arrives
  cancellability        a wrong answer can be stopped without waiting

The last one is the one prototypes skip, and the one users reach for most.
"""

from __future__ import annotations

from dataclasses import dataclass, field
from enum import Enum
from typing import Iterator


class State(str, Enum):
    IDLE = "idle"
    WAITING = "waiting"
    STREAMING = "streaming"
    DONE = "done"
    ERROR = "error"
    CANCELLED = "cancelled"


class StreamFailed(RuntimeError):
    """The upstream stream broke partway through."""


@dataclass
class Frame:
    """One observable moment, as a UI would render it."""

    tick: int
    state: State
    text: str
    note: str = ""

    def line(self) -> str:
        shown = self.text if len(self.text) <= 34 else self.text[:31] + "..."
        base = f"t={self.tick:<3} {self.state.value:<9} {shown!r}"
        return f"{base}  {self.note}" if self.note else base


@dataclass
class Transcript:
    frames: list[Frame] = field(default_factory=list)

    @property
    def final(self) -> Frame:
        return self.frames[-1]

    def time_to_first_token(self) -> int | None:
        """Ticks until the user saw ANY output. None if they never did."""
        for frame in self.frames:
            if frame.state is State.STREAMING and frame.text:
                return frame.tick
        return None

    def states(self) -> list[State]:
        return [f.state for f in self.frames]

    def summary(self) -> str:
        ttft = self.time_to_first_token()
        return (
            f"{self.final.state.value} after {self.final.tick} ticks, "
            f"ttft={ttft if ttft is not None else 'never'}, "
            f"{len(self.final.text)} chars"
        )


def token_source(tokens: list[str], *, fail_at: int | None = None) -> Iterator[str]:
    """Yield tokens, optionally breaking partway through.

    A stream that fails after emitting real content is the interesting case:
    the user has already read something, so discarding it is a worse outcome
    than keeping it and saying the answer is incomplete.
    """
    for index, token in enumerate(tokens):
        if fail_at is not None and index == fail_at:
            raise StreamFailed(f"upstream closed after {index} tokens")
        yield token


def render_stream(
    tokens: list[str],
    *,
    latency_ticks: int = 3,
    fail_at: int | None = None,
    cancel_at: int | None = None,
    keep_partial_on_error: bool = True,
) -> Transcript:
    """Drive the state machine and record every frame a UI would show.

    `latency_ticks` is the wait before the first token -- the part the user
    experiences as the system not having heard them.
    """
    # TASK 1: drive the state machine, appending one Frame per observable
    # moment.
    #   - start at tick 0 with State.IDLE and empty text
    #   - emit `latency_ticks` WAITING frames with empty text; an unchanged
    #     screen is indistinguishable from a broken one
    #   - for each token: advance the tick, APPEND to the accumulated text, and
    #     emit a STREAMING frame (text only ever grows, and every frame differs)
    #   - StopIteration -> a DONE frame
    #   - StreamFailed  -> an ERROR frame keeping the text so far, unless
    #     keep_partial_on_error is False. The user already read it.
    #   - after `cancel_at` tokens -> a CANCELLED frame keeping the text, with
    #     note "stopped by user". Not an error: they stopped it deliberately.
    #   - a terminal frame ENDS the run. Return immediately, or a late token
    #     appends to a response the user already stopped.
    raise NotImplementedError("implement render_stream")

def perceived_wait(transcript: Transcript) -> int:
    """Ticks the user spent with nothing to read.

    This is the number that correlates with how slow an interface feels, and it
    is NOT the total duration. A response that streams for forty ticks after a
    three-tick wait feels faster than one that appears complete after ten.
    """
    # TASK 2: ticks the user spent with nothing to read.
    # That is the time to first token when there is one, and the WHOLE run when
    # there is not. It is not the total duration, and confusing the two is the
    # most common mistake in this area.
    raise NotImplementedError("implement perceived_wait")

def is_terminal(state: State) -> bool:
    return state in (State.DONE, State.ERROR, State.CANCELLED)
tests/run_tests.sh (202 bytes)
#!/usr/bin/env bash
# Run the day 358 streaming UX test suite.
# Offline, standard library plus pytest. No network, no API key.
set -euo pipefail
cd "$(dirname "$0")/.."
python3 -m pytest tests -q "$@"
tests/test_stream_ux.py (4666 bytes)
"""Grouped by UX property, so a failure names what the user would feel.

Run with: bash tests/run_tests.sh
"""

from __future__ import annotations

import os
import sys

import pytest

HERE = os.path.dirname(os.path.abspath(__file__))
LAB = os.path.dirname(HERE)
sys.path.insert(0, os.path.join(LAB, "examples"))

from stream_ux import (  # noqa: E402
    State,
    StreamFailed,
    is_terminal,
    perceived_wait,
    render_stream,
    token_source,
)

TOKENS = ["The ", "refund ", "window ", "is ", "thirty ", "days."]


# ------------------------------------------------------- the state machine


def test_healthy_run_reaches_done():
    t = render_stream(TOKENS)
    assert t.final.state is State.DONE
    assert t.final.text == "".join(TOKENS)


def test_states_occur_in_a_legal_order():
    states = render_stream(TOKENS).states()
    assert states[0] is State.IDLE
    assert State.WAITING in states
    assert states.index(State.WAITING) < states.index(State.STREAMING)
    assert is_terminal(states[-1])


def test_no_frames_follow_a_terminal_state():
    for kwargs in ({}, {"fail_at": 3}, {"cancel_at": 2}):
        t = render_stream(TOKENS, **kwargs)
        terminal = [i for i, f in enumerate(t.frames) if is_terminal(f.state)]
        assert terminal == [len(t.frames) - 1], "a terminal state must end the run"


# ------------------------------------------------------ perceived latency


def test_streaming_is_perceived_as_faster_than_blocking():
    # Same content, same total work. The difference is entirely when the user
    # first has something to read.
    streaming = render_stream(TOKENS, latency_ticks=3)
    blocking = render_stream(TOKENS, latency_ticks=9)
    assert perceived_wait(streaming) < perceived_wait(blocking)


def test_time_to_first_token_ignores_the_waiting_frames():
    t = render_stream(TOKENS, latency_ticks=5)
    assert t.time_to_first_token() == 6  # 5 waiting frames, then the first token


def test_perceived_wait_is_not_total_duration():
    t = render_stream(TOKENS, latency_ticks=2)
    assert perceived_wait(t) < t.final.tick


def test_a_run_that_never_emits_has_no_first_token():
    t = render_stream([], latency_ticks=2)
    assert t.time_to_first_token() is None
    # With nothing ever shown, the whole run is perceived wait.
    assert perceived_wait(t) == t.final.tick


# ----------------------------------------------------- incremental output


def test_text_only_grows_during_streaming():
    t = render_stream(TOKENS)
    lengths = [len(f.text) for f in t.frames if f.state is State.STREAMING]
    assert lengths == sorted(lengths)
    assert len(set(lengths)) == len(lengths), "each frame should add something"


def test_waiting_frames_show_nothing():
    t = render_stream(TOKENS, latency_ticks=4)
    assert all(f.text == "" for f in t.frames if f.state is State.WAITING)


# --------------------------------------------------------- partial output


def test_a_failed_stream_keeps_what_arrived_by_default():
    t = render_stream(TOKENS, fail_at=3)
    assert t.final.state is State.ERROR
    assert t.final.text == "The refund window "
    assert "upstream closed" in t.final.note


def test_discarding_partial_output_loses_what_the_user_read():
    kept = render_stream(TOKENS, fail_at=3)
    discarded = render_stream(TOKENS, fail_at=3, keep_partial_on_error=False)
    assert discarded.final.text == ""
    assert len(kept.final.text) > 0
    # Both failed at the same point; only one wasted the user's reading.
    assert kept.final.tick == discarded.final.tick


def test_failure_before_any_token_leaves_nothing_to_keep():
    t = render_stream(TOKENS, fail_at=0)
    assert t.final.state is State.ERROR
    assert t.final.text == ""
    assert t.time_to_first_token() is None


# --------------------------------------------------------- cancellation


def test_cancellation_stops_early_and_keeps_the_partial_text():
    t = render_stream(TOKENS, cancel_at=2)
    assert t.final.state is State.CANCELLED
    assert t.final.text == "The refund "
    assert t.final.tick < render_stream(TOKENS).final.tick


def test_cancellation_is_not_an_error():
    t = render_stream(TOKENS, cancel_at=2)
    assert t.final.state is not State.ERROR
    assert t.final.note == "stopped by user"


# --------------------------------------------------------- the source


def test_token_source_raises_at_the_configured_point():
    gen = token_source(TOKENS, fail_at=2)
    assert next(gen) == "The "
    assert next(gen) == "refund "
    with pytest.raises(StreamFailed):
        next(gen)


def test_token_source_completes_when_it_does_not_fail():
    assert list(token_source(TOKENS)) == TOKENS

Troubleshooting

Troubleshooting — Day 358

perceived_wait equals the total duration on every run

You are returning transcript.final.tick unconditionally. It should be the time to first token whenever there is one, and only fall back to the final tick when nothing was ever shown. Those are different quantities, and conflating them makes streaming look no better than blocking.

The blocking run does not look worse than the streaming one

You are comparing total duration. Compare perceived_wait, or time_to_first_token. In the reference run both deliver 47 characters and the difference is 4 ticks against 12.

test_no_frames_follow_a_terminal_state fails

Your loop appends the terminal frame and keeps going. Return immediately after appending done, error or cancelled. This is the bug that lets a late token append to a response the user already stopped — a stop button that does not stop.

The error case loses text that should be kept

You are clearing the accumulated text when the stream raises. Keep it unless keep_partial_on_error is False. The user has already read it; the failure does not un-read it.

test_text_only_grows_during_streaming fails on duplicate lengths

Every streaming frame must add something. If two consecutive frames have the same length you are re-emitting the buffer rather than appending a token — usually a frame appended before the token is added rather than after.

time_to_first_token returns a waiting tick

It must find the first frame that is STREAMING and has non-empty text. A waiting frame has empty text by construction, so checking the state alone is not enough if you ever emit an empty streaming frame.

Cancellation is reported as an error

cancel_at should produce State.CANCELLED with the note "stopped by user", keeping the partial text. Reporting it as an error tells the user something broke when they were the one who stopped it.

A run with no tokens behaves strangely

render_stream([]) is a legitimate case: the stream completes without ever emitting. time_to_first_token should return None, and perceived_wait should then be the whole run — the user waited for the entire duration and saw nothing.

NotImplementedError on most tests

Expected. The starter stubs render_stream and perceived_wait — see expected-output/starter-run.txt, which also names the two tests that pass without them.

Security notes

Security notes — Day 358

What this lab does and does not touch

  • No network. The token stream is a generator over a fixed list. No test opens a socket.
  • No credentials. No API key, token or account.
  • No writes outside this directory. Only __pycache__/, .pytest_cache/ and, if you create it, .venv/.
  • No sudo, no global installs.
  • No real clock. Time is a logical tick counter, so the lab cannot be affected by, or affect, system timing.

Why streaming makes escaping harder, and what to do about it

This lab renders plain strings, so nothing here is exploitable. A real interface renders into a document, and incremental rendering changes the threat model in a specific way:

A sanitiser written to process a complete document sees a partial one. A tag that arrives split across two tokens — <scr then ipt> — is not a tag when either half is examined alone. Naive "sanitise the buffer each time" logic can therefore pass content that would have been caught in a single pass over the finished text.

The rule is: escape on every append, and escape the token you are adding rather than re-processing the accumulated buffer. Treat model output as untrusted text throughout, never as markup, and never as HTML you assemble by concatenation.

Whole-response checks conflict with streaming

Streaming means the first tokens are visible before the last ones exist. Any check that needs the complete response — a moderation pass, a policy filter, a schema validation — cannot run before the user has already read part of the answer.

This is a genuine trade-off rather than a bug, and it has three honest resolutions:

  • Block for content that must be checked. Accept the perceived-latency cost where it is warranted.
  • Check incrementally where the check permits it, accepting that a violation may be briefly visible before it is caught.
  • Stream to a holding area the user does not see until a check passes, which is blocking with extra steps and sometimes the clearest way to describe it.

Choose deliberately. Streaming by default, then discovering the moderation pass has nowhere to run, is the failure this note exists to prevent.

Partial and cancelled responses are still content

A partial answer left on screen after an error, and a cancelled response the user stopped reading, were both generated. They may be in your logs, your traces and your cost records. They carry the same retention, privacy and logging obligations as a complete answer — a point that is easy to miss because neither "finished".

Reversing everything this lab did

rm -rf __pycache__ .pytest_cache .venv

Nothing else was created, and nothing outside this directory was modified.