AI Engineering: Agents and Applications › Full-Stack AI Applications › Day 319

Hands-on lab — Day 319: Auth, Quotas, and Billing

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/auth_billing.py

Test

bash tests/run_tests.sh

File tree

examples/auth_billing.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/auth_billing.py
tests/run_tests.sh
tests/test_auth_billing.py
troubleshooting.md

Lab README

Day 319 Lab: Auth, Quotas, and Billing

Lesson

  • Lesson title: Auth, Quotas, and Billing
  • Day number: 319 of 365
  • Lesson article: https://ai-roadmap-365.github.io/day-319-auth-quotas-and-billing
  • Lab files: everything you need is in this directory — follow “How to run” below.
  • Browse the course locally: from the repository root, this lab also appears in the course website at /labs/day-319-auth-quotas-and-billing when the site is running.

Purpose

Build an Auth, Quotas, and Two-Phase Credit Billing Engine in Python supporting SHA-256 API key hashing, RPM/TPM rate limiting, pre-authorization credit holds, and token settlement.

Learning objectives

  • Hash and verify API keys securely.
  • Enforce dual Requests-per-Minute and Tokens-per-Minute limits.
  • Implement credit pre-authorization holds preventing account overdrafts.
  • Settle exact token costs and reconcile tenant balances.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library, Pytest

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/auth_billing.py: Starter implementation skeleton
  • examples/auth_billing.py: Verified reference implementation
  • tests/test_auth_billing.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/auth_billing.py

What the commands do

  • Executes tenant registration, authentication, pre-auth holds, and settlement reconciliation.

Expected output

(True, 'HOLD_RESERVED', 'hold_1')

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Valid authentication and credit pre-authorization hold
  • Rejection on incorrect API key
  • Rejection when available balance is insufficient
  • Rejection when RPM or TPM limits are exceeded
  • Accurate settlement, hold release, and balance deduction

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Ensure reserved_holds is deducted upon settlement.

Security notes

Runs completely offline on local CPU.

Extension exercises

Implement Stripe metered billing event payload emission.

Day number: 319 of 365

Expected output

FIELDS.md

# Measured Values
- `tenants_registered`: 5
- `holds_settled`: 4

examples-run.txt

(True, 'HOLD_RESERVED', 'hold_1')

measured-values.txt

tenants_registered=5
holds_settled=4

run-output.txt

(True, 'HOLD_RESERVED', 'hold_1')

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

self = <examples.auth_billing.AuthBillingEngine object at 0x10ac3e780>
tenant_id = 'org_1', raw_api_key = 'key', initial_credits = 5.0, rpm_limit = 60
tpm_limit = 100000

    def register_tenant(self, tenant_id: str, raw_api_key: str, initial_credits: float=10.0, rpm_limit: int=60, tpm_limit: int=100000):
>       raise NotImplementedError('TASK 1: implement register_tenant.')
E       NotImplementedError: TASK 1: implement register_tenant.

examples/auth_billing.py:19: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_auth_billing.py::test_register_and_pre_auth - NotImplemente...
FAILED tests/test_auth_billing.py::test_invalid_api_key_rejection - NotImplem...
FAILED tests/test_auth_billing.py::test_insufficient_available_balance - NotI...
FAILED tests/test_auth_billing.py::test_rate_limit_rpm_enforcement - NotImple...
FAILED tests/test_auth_billing.py::test_settle_token_usage - NotImplementedEr...
============================== 5 failed in 0.04s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-319-auth-quotas-and-billing
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_auth_billing.py::test_register_and_pre_auth PASSED            [ 20%]
tests/test_auth_billing.py::test_invalid_api_key_rejection PASSED        [ 40%]
tests/test_auth_billing.py::test_insufficient_available_balance PASSED   [ 60%]
tests/test_auth_billing.py::test_rate_limit_rpm_enforcement PASSED       [ 80%]
tests/test_auth_billing.py::test_settle_token_usage PASSED               [100%]

============================== 5 passed in 0.04s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-319-auth-quotas-and-billing
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_auth_billing.py::test_register_and_pre_auth PASSED            [ 20%]
tests/test_auth_billing.py::test_invalid_api_key_rejection PASSED        [ 40%]
tests/test_auth_billing.py::test_insufficient_available_balance PASSED   [ 60%]
tests/test_auth_billing.py::test_rate_limit_rpm_enforcement PASSED       [ 80%]
tests/test_auth_billing.py::test_settle_token_usage PASSED               [100%]

============================== 5 passed in 0.04s ===============================

Source files

examples/auth_billing.py (3534 bytes)
import hashlib
import time
from typing import Dict, Any, Optional, Tuple

class AuthBillingEngine:
    def __init__(self):
        self.tenants: Dict[str, Dict[str, Any]] = {}
        self.active_holds: Dict[str, Dict[str, Any]] = {}
        
    def register_tenant(self, tenant_id: str, raw_api_key: str, initial_credits: float = 10.0, rpm_limit: int = 60, tpm_limit: int = 100000):
        key_hash = hashlib.sha256(raw_api_key.encode()).hexdigest()
        self.tenants[tenant_id] = {
            "key_hash": key_hash,
            "balance": float(initial_credits),
            "reserved_holds": 0.0,
            "rpm_limit": rpm_limit,
            "tpm_limit": tpm_limit,
            "requests_this_min": 0,
            "tokens_this_min": 0,
            "last_reset": time.time()
        }
        
    def authenticate_and_reserve_hold(self, tenant_id: str, raw_api_key: str, estimated_cost: float = 0.02, est_tokens: int = 500) -> Tuple[bool, str, Optional[str]]:
        if tenant_id not in self.tenants:
            return False, "Unauthorized: Unknown Tenant", None
            
        tenant = self.tenants[tenant_id]
        if hashlib.sha256(raw_api_key.encode()).hexdigest() != tenant["key_hash"]:
            return False, "Unauthorized: Invalid API Key", None
            
        now = time.time()
        if now - tenant["last_reset"] > 60:
            tenant["requests_this_min"] = 0
            tenant["tokens_this_min"] = 0
            tenant["last_reset"] = now
            
        if tenant["requests_this_min"] >= tenant["rpm_limit"]:
            return False, "Rate Limit Exceeded: RPM limit reached", None
            
        if tenant["tokens_this_min"] + est_tokens > tenant["tpm_limit"]:
            return False, "Rate Limit Exceeded: TPM limit reached", None
            
        available_balance = tenant["balance"] - tenant["reserved_holds"]
        if available_balance < estimated_cost:
            return False, "Payment Required: Insufficient available credit", None
            
        hold_id = f"hold_{len(self.active_holds)+1}"
        tenant["reserved_holds"] += estimated_cost
        tenant["requests_this_min"] += 1
        tenant["tokens_this_min"] += est_tokens
        
        self.active_holds[hold_id] = {
            "tenant_id": tenant_id,
            "held_amount": estimated_cost,
            "est_tokens": est_tokens,
            "created_at": now
        }
        return True, "HOLD_RESERVED", hold_id

    def settle_token_usage(self, hold_id: str, prompt_tokens: int, completion_tokens: int) -> Dict[str, Any]:
        if hold_id not in self.active_holds:
            return {"status": "ERROR", "message": "Invalid hold ID"}
            
        hold = self.active_holds.pop(hold_id)
        tenant = self.tenants[hold["tenant_id"]]
        
        actual_cost = (prompt_tokens * 3.0 / 1_000_000) + (completion_tokens * 15.0 / 1_000_000)
        actual_cost = round(actual_cost, 6)
        
        tenant["reserved_holds"] -= hold["held_amount"]
        tenant["balance"] -= actual_cost
        
        return {
            "status": "SETTLED",
            "hold_id": hold_id,
            "actual_cost_usd": actual_cost,
            "remaining_balance": round(tenant["balance"], 6),
            "available_balance": round(tenant["balance"] - tenant["reserved_holds"], 6)
        }

if __name__ == "__main__":
    engine = AuthBillingEngine()
    engine.register_tenant("org_1", "key123", 5.0)
    print(engine.authenticate_and_reserve_hold("org_1", "key123", 0.02))
metadata.yml (399 bytes)
lesson_id: D319
day: 319
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/auth_billing.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/auth_billing.py (1341 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

import hashlib
import time
from typing import Dict, Any, Optional, Tuple

class AuthBillingEngine:

    def __init__(self):
        self.tenants: Dict[str, Dict[str, Any]] = {}
        self.active_holds: Dict[str, Dict[str, Any]] = {}

    def register_tenant(self, tenant_id: str, raw_api_key: str, initial_credits: float=10.0, rpm_limit: int=60, tpm_limit: int=100000):
        raise NotImplementedError('TASK 1: implement register_tenant.')

    def authenticate_and_reserve_hold(self, tenant_id: str, raw_api_key: str, estimated_cost: float=0.02, est_tokens: int=500) -> Tuple[bool, str, Optional[str]]:
        raise NotImplementedError('TASK 2: implement authenticate_and_reserve_hold.')

    def settle_token_usage(self, hold_id: str, prompt_tokens: int, completion_tokens: int) -> Dict[str, Any]:
        raise NotImplementedError('TASK 3: implement settle_token_usage.')
if __name__ == '__main__':
    engine = AuthBillingEngine()
    engine.register_tenant('org_1', 'key123', 5.0)
    print(engine.authenticate_and_reserve_hold('org_1', 'key123', 0.02))
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_auth_billing.py (2182 bytes)
import pytest
from examples.auth_billing import AuthBillingEngine

def test_register_and_pre_auth():
    engine = AuthBillingEngine()
    engine.register_tenant("org_alpha", "sk_test_123", initial_credits=10.0)
    ok, msg, hold_id = engine.authenticate_and_reserve_hold("org_alpha", "sk_test_123", estimated_cost=0.05)
    assert ok is True
    assert msg == "HOLD_RESERVED"
    assert hold_id is not None
    assert engine.tenants["org_alpha"]["reserved_holds"] == 0.05

def test_invalid_api_key_rejection():
    engine = AuthBillingEngine()
    engine.register_tenant("org_alpha", "sk_test_123", initial_credits=10.0)
    ok, msg, hold = engine.authenticate_and_reserve_hold("org_alpha", "wrong_key")
    assert ok is False
    assert "Invalid API Key" in msg

def test_insufficient_available_balance():
    engine = AuthBillingEngine()
    engine.register_tenant("org_poor", "key", initial_credits=0.01)
    ok, msg, hold = engine.authenticate_and_reserve_hold("org_poor", "key", estimated_cost=0.05)
    assert ok is False
    assert "Insufficient available credit" in msg

def test_rate_limit_rpm_enforcement():
    engine = AuthBillingEngine()
    engine.register_tenant("org_limited", "key", initial_credits=10.0, rpm_limit=2)
    ok1, _, h1 = engine.authenticate_and_reserve_hold("org_limited", "key", 0.01)
    ok2, _, h2 = engine.authenticate_and_reserve_hold("org_limited", "key", 0.01)
    ok3, msg3, _ = engine.authenticate_and_reserve_hold("org_limited", "key", 0.01)
    assert ok1 is True
    assert ok2 is True
    assert ok3 is False
    assert "RPM limit reached" in msg3

def test_settle_token_usage():
    engine = AuthBillingEngine()
    engine.register_tenant("org_1", "key", initial_credits=5.0)
    _, _, hold_id = engine.authenticate_and_reserve_hold("org_1", "key", estimated_cost=0.05)
    
    # 1000 prompt tokens ($0.003) + 200 completion tokens ($0.003) = $0.006
    settle = engine.settle_token_usage(hold_id, prompt_tokens=1000, completion_tokens=200)
    assert settle["status"] == "SETTLED"
    assert settle["actual_cost_usd"] == 0.006
    assert settle["remaining_balance"] == 4.994
    assert engine.tenants["org_1"]["reserved_holds"] == 0.0

Troubleshooting

Troubleshooting - Day 319\nCheck that hold IDs are properly released on settlement.\n

Security notes

Security - Day 319\nRaw API keys are never stored in plaintext; SHA-256 hashes are used for lookup.\n