AI Engineering: Agents and Applications › Full-Stack AI Applications › Day 322

Hands-on lab — Day 322: A Full-Stack AI Application

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/full_stack_app.py

Test

bash tests/run_tests.sh

File tree

examples/full_stack_app.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/full_stack_app.py
tests/run_tests.sh
tests/test_full_stack_app.py
troubleshooting.md

Lab README

Day 322 Lab: A Full-Stack AI Application

Lesson

Purpose

Build a Complete Full-Stack AI Application Engine in Python integrating authentication, rate limiting, exact caching, two-phase credit holds, and multi-provider failover routing.

Learning objectives

  • Implement multi-tenant authentication and quota verification.
  • Intercept cached queries to deliver instant zero-cost responses.
  • Enforce credit pre-authorization holds preventing negative balance overdrafts.
  • Execute multi-provider fallback routing when upstream services degrade.
  • Reconcile exact token usage charges upon stream completion.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library, Pytest

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/full_stack_app.py: Starter implementation skeleton
  • examples/full_stack_app.py: Verified reference implementation
  • tests/test_full_stack_app.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/full_stack_app.py

What the commands do

  • Simulates the entire end-to-end full-stack request execution lifecycle.

Expected output

Normal: {'status': 'SUCCESS', 'cached': False, 'resolved_provider': 'primary_claude', 'fallback_occurred': False, 'actual_cost_usd': 0.00465, 'remaining_balance': 9.99535, 'response': '[primary_claude] Response for: Hi'}

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Normal full-stack chat execution with credit deduction
  • Zero-cost exact cache response delivery
  • Rejection on insufficient credit balance
  • Seamless secondary provider failover on primary failure
  • Graceful error handling when all upstream providers fail

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Ensure hold amount is released on total upstream failure.

Security notes

Runs completely offline on local CPU.

Extension exercises

Implement dynamic multi-tier semantic caching in the pipeline.

Day number: 322 of 365

Expected output

FIELDS.md

# Measured Values
- `requests_processed`: 12
- `cache_hits`: 2

examples-run.txt

Normal: {'status': 'SUCCESS', 'cached': False, 'resolved_provider': 'primary_claude', 'fallback_occurred': False, 'actual_cost_usd': 0.00465, 'remaining_balance': 9.99535, 'response': '[primary_claude] Response for: Hi'}

measured-values.txt

requests_processed=12
cache_hits=2

run-output.txt

Normal: {'status': 'SUCCESS', 'cached': False, 'resolved_provider': 'primary_claude', 'fallback_occurred': False, 'actual_cost_usd': 0.00465, 'remaining_balance': 9.99535, 'response': '[primary_claude] Response for: Hi'}

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

self = <examples.full_stack_app.FullStackAIAppEngine object at 0x10b04a650>
tenant_id = 'tenant_alpha', raw_key = 'sk_alpha_123', prompt = 'Tell story'
simulated_failing_providers = ['primary_claude', 'secondary_openai', 'backup_vllm']

    def process_chat_request(self, tenant_id: str, raw_key: str, prompt: str, simulated_failing_providers: Optional[List[str]]=None) -> Dict[str, Any]:
>       raise NotImplementedError('TASK 1: implement process_chat_request.')
E       NotImplementedError: TASK 1: implement process_chat_request.

examples/full_stack_app.py:21: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_full_stack_app.py::test_normal_chat_request - NotImplemente...
FAILED tests/test_full_stack_app.py::test_exact_cache_hit - NotImplementedErr...
FAILED tests/test_full_stack_app.py::test_payment_required_insufficient_credit
FAILED tests/test_full_stack_app.py::test_multi_provider_failover - NotImplem...
FAILED tests/test_full_stack_app.py::test_upstream_outage_all_providers_fail
============================== 5 failed in 0.03s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-322-a-full-stack-ai-application
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_full_stack_app.py::test_normal_chat_request PASSED            [ 20%]
tests/test_full_stack_app.py::test_exact_cache_hit PASSED                [ 40%]
tests/test_full_stack_app.py::test_payment_required_insufficient_credit PASSED [ 60%]
tests/test_full_stack_app.py::test_multi_provider_failover PASSED        [ 80%]
tests/test_full_stack_app.py::test_upstream_outage_all_providers_fail PASSED [100%]

============================== 5 passed in 0.02s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-322-a-full-stack-ai-application
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_full_stack_app.py::test_normal_chat_request PASSED            [ 20%]
tests/test_full_stack_app.py::test_exact_cache_hit PASSED                [ 40%]
tests/test_full_stack_app.py::test_payment_required_insufficient_credit PASSED [ 60%]
tests/test_full_stack_app.py::test_multi_provider_failover PASSED        [ 80%]
tests/test_full_stack_app.py::test_upstream_outage_all_providers_fail PASSED [100%]

============================== 5 passed in 0.02s ===============================

Source files

examples/full_stack_app.py (3842 bytes)
import hashlib
import time
from typing import Dict, Any, List, Optional, Tuple

class FullStackAIAppEngine:
    def __init__(self):
        self.tenants: Dict[str, Dict[str, Any]] = {
            "tenant_alpha": {
                "key_hash": hashlib.sha256(b"sk_alpha_123").hexdigest(),
                "balance": 10.0,
                "reserved_holds": 0.0,
                "rpm_limit": 60,
                "requests_this_min": 0,
                "last_reset": time.time()
            },
            "tenant_poor": {
                "key_hash": hashlib.sha256(b"sk_poor_123").hexdigest(),
                "balance": 0.01,
                "reserved_holds": 0.0,
                "rpm_limit": 10,
                "requests_this_min": 0,
                "last_reset": time.time()
            }
        }
        self.cache: Dict[str, str] = {}
        self.active_holds: Dict[str, Dict[str, Any]] = {}
        self.providers = ["primary_claude", "secondary_openai", "backup_vllm"]
        
    def process_chat_request(
        self,
        tenant_id: str,
        raw_key: str,
        prompt: str,
        simulated_failing_providers: Optional[List[str]] = None
    ) -> Dict[str, Any]:
        if tenant_id not in self.tenants:
            return {"status": "AUTH_ERROR", "message": "Invalid tenant ID"}
            
        tenant = self.tenants[tenant_id]
        if hashlib.sha256(raw_key.encode()).hexdigest() != tenant["key_hash"]:
            return {"status": "AUTH_ERROR", "message": "Invalid API key"}
            
        now = time.time()
        if now - tenant["last_reset"] > 60:
            tenant["requests_this_min"] = 0
            tenant["last_reset"] = now
            
        if tenant["requests_this_min"] >= tenant["rpm_limit"]:
            return {"status": "RATE_LIMITED", "message": "RPM limit exceeded"}
            
        cache_key = f"{tenant_id}:{prompt.strip().lower()}"
        if cache_key in self.cache:
            return {
                "status": "SUCCESS",
                "cached": True,
                "cost_usd": 0.0,
                "response": self.cache[cache_key],
                "remaining_balance": round(tenant["balance"], 6)
            }
            
        est_cost = 0.030
        available_balance = tenant["balance"] - tenant["reserved_holds"]
        if available_balance < est_cost:
            return {"status": "PAYMENT_REQUIRED", "message": "Insufficient credit balance"}
            
        hold_id = f"hold_{len(self.active_holds)+1}"
        tenant["reserved_holds"] += est_cost
        tenant["requests_this_min"] += 1
        
        failing = set(simulated_failing_providers or [])
        resolved_provider = None
        attempted_providers = []
        
        for p in self.providers:
            attempted_providers.append(p)
            if p not in failing:
                resolved_provider = p
                break
                
        if not resolved_provider:
            tenant["reserved_holds"] -= est_cost
            return {"status": "UPSTREAM_OUTAGE", "attempted": attempted_providers}
            
        actual_cost = 0.00465
        tenant["reserved_holds"] -= est_cost
        tenant["balance"] -= actual_cost
        
        full_response = f"[{resolved_provider}] Response for: {prompt}"
        self.cache[cache_key] = full_response
        
        return {
            "status": "SUCCESS",
            "cached": False,
            "resolved_provider": resolved_provider,
            "fallback_occurred": len(attempted_providers) > 1,
            "actual_cost_usd": actual_cost,
            "remaining_balance": round(tenant["balance"], 6),
            "response": full_response
        }

if __name__ == "__main__":
    app = FullStackAIAppEngine()
    print("Normal:", app.process_chat_request("tenant_alpha", "sk_alpha_123", "Hi"))
metadata.yml (401 bytes)
lesson_id: D322
day: 322
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/full_stack_app.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/full_stack_app.py (1377 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

import hashlib
import time
from typing import Dict, Any, List, Optional, Tuple

class FullStackAIAppEngine:

    def __init__(self):
        self.tenants: Dict[str, Dict[str, Any]] = {'tenant_alpha': {'key_hash': hashlib.sha256(b'sk_alpha_123').hexdigest(), 'balance': 10.0, 'reserved_holds': 0.0, 'rpm_limit': 60, 'requests_this_min': 0, 'last_reset': time.time()}, 'tenant_poor': {'key_hash': hashlib.sha256(b'sk_poor_123').hexdigest(), 'balance': 0.01, 'reserved_holds': 0.0, 'rpm_limit': 10, 'requests_this_min': 0, 'last_reset': time.time()}}
        self.cache: Dict[str, str] = {}
        self.active_holds: Dict[str, Dict[str, Any]] = {}
        self.providers = ['primary_claude', 'secondary_openai', 'backup_vllm']

    def process_chat_request(self, tenant_id: str, raw_key: str, prompt: str, simulated_failing_providers: Optional[List[str]]=None) -> Dict[str, Any]:
        raise NotImplementedError('TASK 1: implement process_chat_request.')
if __name__ == '__main__':
    app = FullStackAIAppEngine()
    print('Normal:', app.process_chat_request('tenant_alpha', 'sk_alpha_123', 'Hi'))
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_full_stack_app.py (1848 bytes)
import pytest
from examples.full_stack_app import FullStackAIAppEngine

def test_normal_chat_request():
    app = FullStackAIAppEngine()
    res = app.process_chat_request("tenant_alpha", "sk_alpha_123", "Explain gravity")
    assert res["status"] == "SUCCESS"
    assert res["cached"] is False
    assert res["resolved_provider"] == "primary_claude"
    assert res["actual_cost_usd"] == 0.00465
    assert res["remaining_balance"] == 9.99535

def test_exact_cache_hit():
    app = FullStackAIAppEngine()
    res1 = app.process_chat_request("tenant_alpha", "sk_alpha_123", "Explain relativity")
    res2 = app.process_chat_request("tenant_alpha", "sk_alpha_123", "Explain relativity")
    assert res2["status"] == "SUCCESS"
    assert res2["cached"] is True
    assert res2["cost_usd"] == 0.0
    assert res2["remaining_balance"] == res1["remaining_balance"]

def test_payment_required_insufficient_credit():
    app = FullStackAIAppEngine()
    res = app.process_chat_request("tenant_poor", "sk_poor_123", "Complex prompt")
    assert res["status"] == "PAYMENT_REQUIRED"
    assert "Insufficient credit" in res["message"]

def test_multi_provider_failover():
    app = FullStackAIAppEngine()
    res = app.process_chat_request("tenant_alpha", "sk_alpha_123", "Tell story", simulated_failing_providers=["primary_claude"])
    assert res["status"] == "SUCCESS"
    assert res["resolved_provider"] == "secondary_openai"
    assert res["fallback_occurred"] is True

def test_upstream_outage_all_providers_fail():
    app = FullStackAIAppEngine()
    res = app.process_chat_request("tenant_alpha", "sk_alpha_123", "Tell story", simulated_failing_providers=["primary_claude", "secondary_openai", "backup_vllm"])
    assert res["status"] == "UPSTREAM_OUTAGE"
    # Ensure hold was released
    assert app.tenants["tenant_alpha"]["reserved_holds"] == 0.0

Troubleshooting

Troubleshooting - Day 322\nConfirm credit holds are released if all upstream providers fail.\n

Security notes

Security - Day 322\nAll requests are authenticated with hashed API credentials and quota-gated.\n