AI Engineering: Agents and Applications › Full-Stack AI Applications › Day 319

Day 319: Auth, Quotas, and Billing

Day 319 of 365 — Auth, Quotas, and Billing

Master multi-tenant authentication, token-level billing, credit pre-authorization, and tiered rate limiting: design Stripe meter integration, token-budget enforcement, and tenant isolation in Python.

Course
AI Engineering: Agents and Applications
Category
Full-Stack AI Applications
Reading time
≈ 35 min
Practical time
≈ 50 min
Lesson duration
1h 25m
Last verified
2026-08-30

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/ai-engineering/day-319-auth-quotas-and-billing

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/ai-engineering/day-319-auth-quotas-and-billing
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

Deploying a generative AI product to the public without robust authentication, rate limiting, and credit governance is a recipe for instant financial disaster. Malicious actors or runaway recursive agent scripts can consume thousands of dollars in GPU token bills in minutes.

In this lesson, we master Multi-Tenant Authentication, Token Quotas, and Two-Phase Credit Settlement: the infrastructure that protects your AI application from fraud and powers usage-based monetization.

Two-Phase Credit Authorization and Settlement Flow showing Phase 1 Pre-Auth Reserve Hold, LLM Inference Stream, and Phase 2 Exact Token Settlement and Hold Release

The idea in plain language

Diagram: request-rate limiting compared with dual request and token limiting

Diagram: the three phases of a two-phase credit settlement for a generation

Think of AI token billing like pumping gasoline at a self-service gas station:

Historical background

Diagram: the progression from flat-rate subscriptions to metered token economics

SaaS monetization and API authentication progressed through three major generations:

1. Flat-Rate Subscription Tiering (2010–2020)

Users paid $20/month for unlimited CRUD database operations. Computational cost per user was fractions of a cent, making flat-rate pricing sustainable.

2. The Early LLM Cost Crisis (2022–2023)

Startups offered flat $20/month access to frontier LLMs. Power users generated millions of tokens per day, causing companies to lose $100+ per month per subscriber.

3. Modern Metered Token Economics (2024–Present)

Modern AI companies implement hybrid billing: a monthly base platform fee paired with metered token usage credits, enforced via real-time two-phase credit reservations (Stripe Metering, Lago, Octane).

What it is — and what it is not

Let us define the core components of AI billing and quota infrastructure:

What AI Auth & Billing Is:

What AI Auth & Billing Is Not:

+-------------------------------------------------------------------------------+
|                      TOKEN QUOTA & RATE LIMITING SCHEMAS                      |
+-------------------+-----------------------+-----------------------------------+
| Metric / Field    | Description           | Unit & Example Value              |
+-------------------+-----------------------+-----------------------------------+
| `tenant_id`       | Organization UUID     | `org_94820a`                      |
| `api_key_hash`    | SHA-256 Secret Hash   | `e3b0c44298fc1c149afbf4c8996fb9242`|
| `credit_balance`  | Prepaid balance in USD| `$48.5000`                        |
| `reserved_holds`  | Active pre-auth holds | `$0.0300`                         |
| `available_credit`| Balance minus holds   | `$48.4700`                        |
| `rpm_limit`       | Requests per minute   | `60 req/min`                      |
| `tpm_limit`       | Tokens per minute     | `150,000 tok/min`                 |
+-------------------+-----------------------+-----------------------------------+

Why it was created and what problems it solves

Two-phase token billing and quota systems solve the four primary financial and security vulnerabilities of generative AI platforms:

1. Eliminating Negative Balance Exposure

Without pre-authorization, a tenant with $0.01 balance can submit 20 concurrent requests generating 4,000 tokens each ($0.50 cost each), plunging their account into negative balance and leaving the platform with unpaid cloud provider bills.

2. Preventing Token Flooding Attacks

Dual rate limiting (RPM + TPM) prevents attackers from sending a few massive 100k-token prompts that overwhelm model capacity without tripping request-count rate limiters.

3. Transparent Metered Invoicing

Tracking exact prompt, completion, and cached token counts enables transparent audit logs and seamless integration with Stripe Metered Billing.

How it works

Let us inspect the complete Multi-Tenant Quota & Tier Enforcement Matrix:

+-------------------------------------------------------------------------------+
|                     TIERED AI RESOURCE ALLOCATION MATRIX                      |
+-------------------+-------------------+-------------------+-------------------+
| Feature Dimension | Free Tier         | Pro Tier          | Enterprise Tier   |
+-------------------+-------------------+-------------------+-------------------+
| Monthly Price     | $0 / month        | $29 / month       | Custom Invoiced   |
| Base Credits      | $1.00 one-time    | $50.00 / month    | Pay-as-you-go net30|
| Rate Limit (RPM)  | 10 requests / min | 120 requests / min| 1,000+ req / min  |
| Throughput (TPM)  | 20,000 tokens/min | 300,000 tokens/min| 2,000,000+ tok/min|
| Max Context / Req | 4,000 tokens      | 32,000 tokens     | 200,000 tokens    |
| Available Models  | Compact / Haiku   | Frontier / Sonnet | Custom vLLM cluster|
+-------------------+-------------------+-------------------+-------------------+

Multi-Tenant Tier Quota Enforcement Matrix comparing Free Tier (10 RPM, 20k TPM, Mini models), Pro Tier (100 RPM, 200k TPM, Sonnet models), and Enterprise Tier (Unlimited RPM, Custom TPM, Dedicated vLLM)

An everyday analogy

Diagram: a hotel credit hold as an analogy for two-phase token billing

Think of two-phase token billing like staying at a luxury hotel:

Examples in practice

Let us build a complete Python Auth, Quotas, and Two-Phase Billing Engine supporting API key hashing, credit pre-authorization holds, and token settlement:

import hashlib
import time
from typing import Dict, Any, Optional, Tuple

class AuthBillingEngine:
    def __init__(self):
        self.tenants: Dict[str, Dict[str, Any]] = {}
        self.active_holds: Dict[str, Dict[str, Any]] = {}
        
    def register_tenant(self, tenant_id: str, raw_api_key: str, initial_credits: float = 10.0, rpm_limit: int = 60, tpm_limit: int = 100000):
        key_hash = hashlib.sha256(raw_api_key.encode()).hexdigest()
        self.tenants[tenant_id] = {
            "key_hash": key_hash,
            "balance": float(initial_credits),
            "reserved_holds": 0.0,
            "rpm_limit": rpm_limit,
            "tpm_limit": tpm_limit,
            "requests_this_min": 0,
            "tokens_this_min": 0,
            "last_reset": time.time()
        }
        
    def authenticate_and_reserve_hold(self, tenant_id: str, raw_api_key: str, estimated_cost: float = 0.02, est_tokens: int = 500) -> Tuple[bool, str, Optional[str]]:
        if tenant_id not in self.tenants:
            return False, "Unauthorized: Unknown Tenant", None
            
        tenant = self.tenants[tenant_id]
        if hashlib.sha256(raw_api_key.encode()).hexdigest() != tenant["key_hash"]:
            return False, "Unauthorized: Invalid API Key", None
            
        now = time.time()
        if now - tenant["last_reset"] > 60:
            tenant["requests_this_min"] = 0
            tenant["tokens_this_min"] = 0
            tenant["last_reset"] = now
            
        if tenant["requests_this_min"] >= tenant["rpm_limit"]:
            return False, "Rate Limit Exceeded: RPM limit reached", None
            
        if tenant["tokens_this_min"] + est_tokens > tenant["tpm_limit"]:
            return False, "Rate Limit Exceeded: TPM limit reached", None
            
        available_balance = tenant["balance"] - tenant["reserved_holds"]
        if available_balance < estimated_cost:
            return False, "Payment Required: Insufficient available credit", None
            
        # Place Hold
        hold_id = f"hold_{len(self.active_holds)+1}"
        tenant["reserved_holds"] += estimated_cost
        tenant["requests_this_min"] += 1
        tenant["tokens_this_min"] += est_tokens
        
        self.active_holds[hold_id] = {
            "tenant_id": tenant_id,
            "held_amount": estimated_cost,
            "est_tokens": est_tokens,
            "created_at": now
        }
        return True, "HOLD_RESERVED", hold_id

    def settle_token_usage(self, hold_id: str, prompt_tokens: int, completion_tokens: int) -> Dict[str, Any]:
        if hold_id not in self.active_holds:
            return {"status": "ERROR", "message": "Invalid hold ID"}
            
        hold = self.active_holds.pop(hold_id)
        tenant = self.tenants[hold["tenant_id"]]
        
        # Calculate exact cost: $3/M prompt, $15/M completion
        actual_cost = (prompt_tokens * 3.0 / 1_000_000) + (completion_tokens * 15.0 / 1_000_000)
        actual_cost = round(actual_cost, 6)
        
        # Release hold and deduct actual cost
        tenant["reserved_holds"] -= hold["held_amount"]
        tenant["balance"] -= actual_cost
        
        return {
            "status": "SETTLED",
            "hold_id": hold_id,
            "actual_cost_usd": actual_cost,
            "remaining_balance": round(tenant["balance"], 6),
            "available_balance": round(tenant["balance"] - tenant["reserved_holds"], 6)
        }

if __name__ == "__main__":
    engine = AuthBillingEngine()
    engine.register_tenant("org_1", "secret_key_123", initial_credits=5.0)
    ok, msg, hold = engine.authenticate_and_reserve_hold("org_1", "secret_key_123", estimated_cost=0.03)
    print("Pre-Auth:", ok, msg, hold)
    settle = engine.settle_token_usage(hold, prompt_tokens=800, completion_tokens=150)
    print("Settled:", settle)

Implications: security, privacy, performance, scalability, and cost

Diagram: values a billing system must compute server-side rather than accept from a client

1. Atomic Balances and Distributed Locks

In high-concurrency multi-instance architectures, storing credit balances in plain Python variables causes race conditions. Use Redis Lua scripts or PostgreSQL SELECT ... FOR UPDATE row locks to ensure atomic balance checks.

2. Webhook Event Delivery to Stripe

When settling token usage, dispatch usage records asynchronously to Stripe Metered Billing via a background queue. Ensure all webhook calls include an idempotency key to prevent duplicate invoice line items.

Production Token Billing Case Study: High-Concurrency Credit Management

In large-scale AI applications serving enterprise organizations with hundreds of concurrent users, managing shared credit wallets without race conditions requires careful database architecture.

+-------------------------------------------------------------------------------+
|                    CONCURRENT CREDIT PRE-AUTH HOLD METRICS                    |
+-------------------+-----------------------+-----------------------------------+
| Metric Dimension  | Naive Database Update | Atomic Redis Lua / Postgres Lock  |
+-------------------+-----------------------+-----------------------------------+
| Race Condition    | High: Overdrafts on   | Zero: Strictly serializable       |
| Risk              | simultaneous calls    | atomic balance deductions         |
| Transaction Time  | 45ms per SQL UPDATE   | 0.8ms Redis Lua execution         |
| Throughput Limit  | 250 requests / sec    | 45,000 operations / sec           |
| Audit Traceability| Partial logging       | 100% immutable ledger event logs  |
+-------------------+-----------------------+-----------------------------------+

1. Atomic Redis Lua Script for Pre-Authorization Holds

To verify credit availability and place a pre-authorization hold atomically in a single network round-trip, production gateways deploy Redis Lua scripts:

Diagram: the requirements of an atomic credit pre-authorisation

-- Atomic Token Hold Reservation Script (reserve_hold.lua)
local tenant_key = KEYS[1]
local hold_id = ARGV[1]
local estimated_cost = tonumber(ARGV[2])
local max_rpm = tonumber(ARGV[3])

local balance = tonumber(redis.call('HGET', tenant_key, 'balance') or '0')
local reserved = tonumber(redis.call('HGET', tenant_key, 'reserved_holds') or '0')
local current_rpm = tonumber(redis.call('HGET', tenant_key, 'rpm_counter') or '0')

if current_rpm >= max_rpm then
    return {0, 'RATE_LIMIT_RPM'}
end

if (balance - reserved) < estimated_cost then
    return {0, 'INSUFFICIENT_CREDIT'}
end

-- Place Hold
redis.call('HINCRBYFLOAT', tenant_key, 'reserved_holds', estimated_cost)
redis.call('HINCRBY', tenant_key, 'rpm_counter', 1)
redis.call('HSET', 'hold:' .. hold_id, 'tenant', tenant_key, 'amount', estimated_cost)

return {1, 'HOLD_ACQUIRED'}

2. Reconciliation with Stripe Metered Usage API

When generations complete, the gateway pushes settlement records into a persistent Kafka or RabbitMQ event queue. Dedicated billing synchronization workers batch these events and dispatch usage meters to Stripe’s Usage Records API (/v1/subscription_items/id/usage_records) using daily idempotency keys to ensure zero double-billing errors.

Diagram: the reconciliation pipeline from settlement to metered invoicing

Production Failure Mode Taxonomy: Multi-Tenant Billing and Quota Vulnerabilities

Operating an AI API platform requires rigorous defense against financial fraud and quota race conditions:

+-------------------------------------------------------------------------------+
|                       BILLING & QUOTA FAILURE TAXONOMY                        |
+-------------------+-----------------------+-----------------------------------+
| Vulnerability     | Attack / Root Cause   | Platform Defense Architecture     |
+-------------------+-----------------------+-----------------------------------+
| Concurrent Token  | Attacker launches 50  | Atomic Redis Lua pre-auth holds   |
| Overdraft Flood   | parallel 4k-token reqs| lock funds before compute starts  |
| Plaintext API Key | Database backup leak  | Store only SHA-256 / bcrypt key   |
| Exposure          | exposes secret keys   | hashes; show key once on creation |
| Token Bucket      | Fixed minute window   | Sliding log algorithm smooths     |
| Burst Spikes      | allows 2x burst limit | rate limits continuously          |
| Invoicing Skew    | Drift between provider| Reconcile provider response usage |
|                   | bills and client ledger| payloads directly in audit table  |
+-------------------+-----------------------+-----------------------------------+

Billing Infrastructure Verification Checklist

  1. API keys are validated using constant-time string comparison algorithms (hmac.compare_digest) to prevent timing attacks.
  2. Pre-authorization holds are placed atomically in single-round-trip database operations.
  3. Every token deduction is logged to an immutable PostgreSQL ledger table with timestamp and prompt hash.
  4. Tenant rate limits support separate burst allowances and steady-state limits.

Diagram: the four verification checks for AI billing infrastructure

Real-World Billing Deep Dive: Tiered Rate Limiting and Token Accounting

To implement robust multi-tenant monetization, platforms enforce hierarchical token quotas across organizational workspaces, developer accounts, and individual end users:

+-------------------------------------------------------------------------------+
|                    HIERARCHICAL QUOTA ENFORCEMENT HIERARCHY                   |
+-------------------+-----------------------+-----------------------------------+
| Governance Level  | Scope & Key Format    | Enforced Rate Limits & Quotas     |
+-------------------+-----------------------+-----------------------------------+
| Enterprise Org    | `org:94820:limits`    | Global Monthly Budget: $10,000/mo |
| Workspace / Team  | `team:finance:limits` | 500 RPM | 1,000,000 TPM           |
| Developer Key     | `key:dev_alice:limits`| 60 RPM | 150,000 TPM | Max $50/day|
| End-User Session  | `user:bob:limits`     | 10 RPM | 20,000 TPM | 5 streams   |
+-------------------+-----------------------+-----------------------------------+

Token Cost Accounting and Reconciliation Pipeline

  1. Prompt & Completion Price Tables: Maintain a versioned model pricing table with micro-dollar costs (e.g. Claude 3.5 Sonnet: $3.00/M prompt, $15.00/M completion; GPT-4o-mini: $0.15/M prompt, $0.60/M completion).
  2. Hold Idempotency and TTL: Active pre-authorization holds must expire automatically after 120 seconds if the backend worker crashes, ensuring funds are never permanently locked in holding state.
  3. Usage Anomaly Detection: Trigger automated security alerts if a single tenant consumes more than 80% of their daily token allowance in under 5 minutes.

Diagram: the components of a token cost accounting pipeline

Alternatives: free, open source, and commercial

Tool / PlatformTypeBest Suited ForKey Capabilities
Stripe Metered BillingCommercialSaaS credit cards & subscriptionsAutomated invoicing, usage aggregation
LagoOpen SourceSelf-hosted usage billingReal-time events, customer wallet balances
LiteLLM Key ManagementOpen SourceAI Gateway Auth & QuotasVirtual API keys, TPM/RPM limits
Redis Token BucketOpen SourceHigh-throughput rate limitingSub-millisecond sliding window counters
+-----------------------+-----------------------+-----------------------+
| Dimension             | Naive Pre-Pay         | Two-Phase Token Auth  |
+-----------------------+-----------------------+-----------------------+
| Balance Overdraft Risk| High (Concurrent run) | Zero (Locked pre-auth)|
| Cost Accuracy         | Crude guess           | Micro-dollar precision|
| Rate Limiting Depth   | Request-count only    | RPM + TPM Dual gating |
| Audit Trail           | Missing               | Full token settlement |
+-----------------------+-----------------------+-----------------------+

When to use it — and when not to

Diagram: when two-phase credit holds and quotas are required

Implement Two-Phase Billing & Quotas for:

Do NOT require two-phase credit holds for:

The AI connection

Knowledge check

  1. Why is Credit Pre-Authorization necessary in AI billing? To reserve an estimated credit hold before generation, preventing tenants from over-drafting balances during expensive LLM calls.
  2. What is the difference between RPM and TPM? RPM measures HTTP request frequency, while TPM measures total token computational throughput.
  3. How does API key hashing protect platform security? By storing one-way cryptographic hashes (SHA-256) in the database, ensuring that compromised database backups cannot leak usable API keys.

Hands-on exercise

In this lab, you will build an Auth, Quotas, and Two-Phase Billing Engine in Python. Your implementation will:

  1. Register tenants with SHA-256 hashed API keys and credit balances.
  2. Verify authentication, enforce dual RPM/TPM rate limits, and place credit pre-authorization holds.
  3. Reject requests when available balance (balance minus active holds) is insufficient.
  4. Settle token usage, deduct exact actual costs, release holds, and report updated balances.

Expected output

============================= test session starts ==============================
collected 5 items

tests/test_auth_billing.py .....                                         [100%]

============================== 5 passed in 0.02s ===============================
[AUTH] API Key validated via SHA-256 hash.
[PRE-AUTH] Reserved $0.030 hold on tenant org_1.
[SETTLEMENT] Settled 950 tokens ($0.00465). Hold released.
[BALANCE] Available balance updated: $4.99535.

Validate your work

Run the pytest test suite:

pytest tests/run_tests.sh

Troubleshooting

  1. Available balance check: Ensure available_balance = balance - reserved_holds is evaluated before granting holds.
  2. Hold release: Verify that reserved_holds is decremented by the exact held amount on settlement.

Common mistakes

Practice assignment

Add Stripe Webhook Event Simulation: format and log a JSON metered event payload (customer_id, timestamp, token_count, usd_amount) upon settlement.

Extension challenge

Implement a Sliding Window Token Bucket Algorithm using microsecond timestamps to provide smooth, sub-minute rate limit recovery.

Quiz

Q1. Why is 'Credit Pre-Authorization' (Two-Phase Token Billing) necessary for generative AI API gateways?

  1. Because SQL databases cannot store any numbers at all
  2. Because credit cards only work in integers
  3. A hold stops a tenant spending beyond their balance
  4. Because LLMs do not accept US Dollars
Show answer

Answer: C. A hold stops a tenant spending beyond their balance

Pre-authorizing funds prevents bankrupt or malicious tenants from incurring unrecoverable GPU costs.

Q2. What is the difference between Requests Per Minute (RPM) and Tokens Per Minute (TPM) rate limiting?

  1. They are identical metrics with different names
  2. TPM is only used in cryptocurrency
  3. RPM applies only to mobile apps
  4. RPM caps calls; TPM caps tokens
Show answer

Answer: D. RPM caps calls; TPM caps tokens

RPM protects network gateway connections while TPM governs actual computational token throughput.

Q3. How should API keys be stored in a production AI SaaS database?

  1. As one-way hashes, with only a prefix shown
  2. As plain text in a public table
  3. In a browser cookie
  4. In a git commit message
Show answer

Answer: A. As one-way hashes, with only a prefix shown

API keys must always be hashed before storage to prevent database leak exposure.

Q4. What occurs during the 'Settlement Phase' of token billing?

  1. The server deletes all user messages
  2. Real usage is settled and the hold is released
  3. The user is logged out
  4. The model is fine-tuned
Show answer

Answer: B. Real usage is settled and the hold is released

Settlement reconciles exact token usage, releases holds, and records metered billing.

Q5. What HTTP status code should an AI gateway return when a tenant exceeds their monthly token quota or runs out of credits?

  1. 200 OK
  2. 500 Internal Server Error only
  3. 402, or 429 with Retry-After
  4. 404 Not Found
Show answer

Answer: C. 402, or 429 with Retry-After

HTTP 402 signals credit exhaustion while 429 signals rate limit exhaustion.

Glossary

Credit Pre-Authorization
Reserving an estimated credit hold before LLM invocation to prevent unpaid token consumption.
Settlement
Reconciling exact token counts, deducting final costs, and releasing unused reserved holds.
Tokens Per Minute (TPM)
Rate limiting metric governing total token throughput per minute per tenant.
Requests Per Minute (RPM)
Rate limiting metric governing total HTTP request frequency per minute.
Metered Billing
Charging customers asynchronously based on actual units of consumption (tokens, tool calls).
API Key Hashing
Storing one-way cryptographic hashes (SHA-256) of API secret keys in the database.
Tenant Isolation
Partitioning data, memory, and billing records strictly by organization ID.
Tiered Quota
Allocating different RPM, TPM, and model access limits based on subscription tier (Free, Pro, Enterprise).

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.