LLMs and Generative AI › The LLM Landscape › Day 242

Hands-on lab — Day 242: Open Weights versus Closed APIs

Commands

Setup

pip install -r requirements/requirements.txt

Run

python3 examples/open_weights_versus_closed_apis_lib.py

Test

./tests/run_tests.sh

File tree

conftest.py
examples/open_weights_versus_closed_apis_lib.py
examples/test_open_weights_versus_closed_apis_lib.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/starter-run.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/open_weights_versus_closed_apis_lib.py
starter/test_open_weights_versus_closed_apis_lib.py
tests/run_tests.sh
tests/test_open_weights_versus_closed_apis_lib.py
troubleshooting.md

Lab README

Lab: Day 242 -- Open Weights versus Closed APIs

Lesson

Day number: 242 of 365. Course: Course06-SS01 (LLMs and Generative AI - Working with LLMs). Topic: Open Weights vs. Closed APIs, Total Cost of Ownership (TCO), and vLLM Serving.

Purpose

Build and test an automated Total Cost of Ownership (TCO) and inference throughput simulator in Python. Calculate monthly break-even thresholds, evaluate high-volume vs low-volume enterprise deployment scenarios, and determine the optimal infrastructure choice.

Learning objectives

  • Formulate the TCO break-even equation comparing GPU server rental against API token charges.
  • Evaluate deployment decisions based on daily token volume thresholds.
  • Understand the operational role of vLLM PagedAttention and continuous batching.
  • Assess data sovereignty and regulatory compliance constraints.

Prerequisites

  • Day 241 (The Model Landscape: Claude, GPT, Gemini, Llama).
  • Python 3.11+ with PyTorch and NumPy.

Supported operating systems

  • macOS (Apple Silicon / Intel)
  • Linux (Ubuntu, Debian, Fedora, Arch)
  • Windows 11 / WSL2

Hardware requirements

  • 1+ CPU cores.
  • 1 GB RAM.
  • 100 MB disk space.

Required software

  • Python 3.11 or newer.
  • pip package manager.
  • virtualenv or venv module.

Free and open-source options

Python and standard math libraries are open source.

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/open_weights_versus_closed_apis_lib.py: Student scaffold file.
  • examples/open_weights_versus_closed_apis_lib.py: Complete reference implementation.
  • tests/test_open_weights_versus_closed_apis_lib.py: Pytest automated validation suite.
  • expected-output/: Verified output logs and baseline values.

How to run

Execute the reference demonstration script:

python3 examples/open_weights_versus_closed_apis_lib.py

What the commands do

  • Computes daily break-even token thresholds.
  • Evaluates deployment recommendations across volume scenarios.
  • Runs unit test assertions.

Expected output

TCO Demo: Break-Even = 247.56M Tok/Day, Low Vol = Closed API, High Vol = Self-Hosted

Validation steps

  1. Verify calculate_break_even computes exact break-even daily tokens.
  2. Confirm evaluate_decision recommends Closed API for volumes below break-even.
  3. Confirm evaluate_decision recommends Self-Hosted for volumes above break-even.
  4. Ensure all unit test assertions pass.

Tests

Run the test runner script:

./tests/run_tests.sh

Cleanup

find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +

Troubleshooting

  • Break-even scaling error: Ensure monthly token totals are converted to daily averages by dividing by 30.

Security notes

All TCO simulations execute locally in process memory.

Extension exercises

  1. Implement an automated GPU cluster reservation amortization tool.
  2. Build a latency simulator measuring tokens/second under varying batch sizes.

Expected output

FIELDS.md

# Expected Output Fields: Day 242

- `Break-Even`: Daily token volume threshold where self-hosting breaks even with closed APIs.
- `Low Vol`: Infrastructure recommendation for low query traffic.
- `High Vol`: Infrastructure recommendation for high query traffic.

examples-run.txt

TCO Demo: Break-Even = 247.56M Tok/Day, Low Vol = Closed API, High Vol = Self-Hosted

measured-values.txt

Break-Even: 247.56M Tok/Day
Low Vol: Closed API
High Vol: Self-Hosted

starter-run.txt

Starter scaffold executed. Ready for student implementation.

test-run.txt

============================= test session starts ==============================
collected 3 items

tests/test_open_weights_versus_closed_apis_lib.py::test_break_even_calculation PASSED [ 33%]
tests/test_open_weights_versus_closed_apis_lib.py::test_decision_evaluation PASSED [ 66%]
tests/test_open_weights_versus_closed_apis_lib.py::test_zero_mlops_cost_break_even PASSED [100%]

============================== 3 passed in 0.40s ===============================

Source files

conftest.py (585 bytes)
"""Make this lab importable when the suite runs.

pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""

import os
import sys

LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
    if os.path.isdir(candidate) and candidate not in sys.path:
        sys.path.insert(0, candidate)
examples/open_weights_versus_closed_apis_lib.py (1433 bytes)
from typing import Dict, Any

class TCOCalculator:
    def __init__(self, gpu_hourly_cost: float = 24.0, mlops_monthly_cost: float = 5000.0):
        self.gpu_hourly_cost = gpu_hourly_cost
        self.mlops_monthly_cost = mlops_monthly_cost

    def calculate_break_even(self, api_rate_per_million: float) -> float:
        fixed_monthly = (self.gpu_hourly_cost * 24 * 30) + self.mlops_monthly_cost
        monthly_tokens = (fixed_monthly / api_rate_per_million) * 1e6
        daily_tokens = monthly_tokens / 30.0
        return round(daily_tokens, 2)

    def evaluate_decision(self, daily_tokens_millions: float, api_rate_per_million: float) -> str:
        monthly_tokens = daily_tokens_millions * 1e6 * 30
        api_monthly = (monthly_tokens / 1e6) * api_rate_per_million
        self_host_monthly = (self.gpu_hourly_cost * 24 * 30) + self.mlops_monthly_cost
        return "Self-Hosted" if self_host_monthly < api_monthly else "Closed API"

def run_tco_demo():
    calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
    be_daily = calc.calculate_break_even(3.00) # $3/M API

    dec_low = calc.evaluate_decision(10.0, 3.00) # 10M tokens/day
    dec_high = calc.evaluate_decision(500.0, 3.00) # 500M tokens/day

    print(f"TCO Demo: Break-Even = {be_daily/1e6:.2f}M Tok/Day, Low Vol = {dec_low}, High Vol = {dec_high}")
    return be_daily, dec_low, dec_high

if __name__ == "__main__":
    run_tco_demo()
examples/test_open_weights_versus_closed_apis_lib.py (1139 bytes)
import pytest
from examples.open_weights_versus_closed_apis_lib import TCOCalculator

def test_break_even_calculation():
    # 8x H100 node: $24/hr * 720 = $17,280 + $5,000 MLOps = $22,280 / mo
    # API: $3.00 / M -> $22,280 / 0.000003 = 7,426,666,667 tokens / mo
    # Daily: ~247,555,555 tokens / day (~247.56M)
    calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
    be = calc.calculate_break_even(3.00)
    assert be == pytest.approx(247555555.56, rel=1e-3)

def test_decision_evaluation():
    calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
    
    # 10M daily -> API is vastly cheaper ($900 vs $22,280)
    assert calc.evaluate_decision(10.0, 3.00) == "Closed API"

    # 500M daily -> Self-hosting is vastly cheaper ($45,000 vs $22,280)
    assert calc.evaluate_decision(500.0, 3.00) == "Self-Hosted"

def test_zero_mlops_cost_break_even():
    calc = TCOCalculator(gpu_hourly_cost=10.0, mlops_monthly_cost=0.0)
    # Fixed = $7,200/mo. API = $2.00/M. Monthly tokens = 3.6B -> Daily = 120M
    be = calc.calculate_break_even(2.00)
    assert be == pytest.approx(120000000.0, rel=1e-3)
metadata.yml (444 bytes)
lesson_id: D242
day: 242
kind: lab
languages:
  - python
setup_commands:
  - 'pip install -r requirements/requirements.txt'
run_commands:
  - 'python3 examples/open_weights_versus_closed_apis_lib.py'
test_commands:
  - './tests/run_tests.sh'
cleanup_commands:
  - 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/open_weights_versus_closed_apis_lib.py (552 bytes)
from typing import Dict, Any

class TCOCalculator:
    def __init__(self, gpu_hourly_cost: float = 24.0, mlops_monthly_cost: float = 5000.0):
        self.gpu_hourly_cost = gpu_hourly_cost
        self.mlops_monthly_cost = mlops_monthly_cost

    def calculate_break_even(self, api_rate_per_million: float) -> float:
        # TODO: Calculate break-even daily token volume
        pass

    def evaluate_decision(self, daily_tokens_millions: float, api_rate_per_million: float) -> str:
        # TODO: Return "Closed API" or "Self-Hosted"
        pass
starter/test_open_weights_versus_closed_apis_lib.py (1139 bytes)
import pytest
from examples.open_weights_versus_closed_apis_lib import TCOCalculator

def test_break_even_calculation():
    # 8x H100 node: $24/hr * 720 = $17,280 + $5,000 MLOps = $22,280 / mo
    # API: $3.00 / M -> $22,280 / 0.000003 = 7,426,666,667 tokens / mo
    # Daily: ~247,555,555 tokens / day (~247.56M)
    calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
    be = calc.calculate_break_even(3.00)
    assert be == pytest.approx(247555555.56, rel=1e-3)

def test_decision_evaluation():
    calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
    
    # 10M daily -> API is vastly cheaper ($900 vs $22,280)
    assert calc.evaluate_decision(10.0, 3.00) == "Closed API"

    # 500M daily -> Self-hosting is vastly cheaper ($45,000 vs $22,280)
    assert calc.evaluate_decision(500.0, 3.00) == "Self-Hosted"

def test_zero_mlops_cost_break_even():
    calc = TCOCalculator(gpu_hourly_cost=10.0, mlops_monthly_cost=0.0)
    # Fixed = $7,200/mo. API = $2.00/M. Monthly tokens = 3.6B -> Daily = 120M
    be = calc.calculate_break_even(2.00)
    assert be == pytest.approx(120000000.0, rel=1e-3)
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 242 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_open_weights_versus_closed_apis_lib.py (1139 bytes)
import pytest
from examples.open_weights_versus_closed_apis_lib import TCOCalculator

def test_break_even_calculation():
    # 8x H100 node: $24/hr * 720 = $17,280 + $5,000 MLOps = $22,280 / mo
    # API: $3.00 / M -> $22,280 / 0.000003 = 7,426,666,667 tokens / mo
    # Daily: ~247,555,555 tokens / day (~247.56M)
    calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
    be = calc.calculate_break_even(3.00)
    assert be == pytest.approx(247555555.56, rel=1e-3)

def test_decision_evaluation():
    calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
    
    # 10M daily -> API is vastly cheaper ($900 vs $22,280)
    assert calc.evaluate_decision(10.0, 3.00) == "Closed API"

    # 500M daily -> Self-hosting is vastly cheaper ($45,000 vs $22,280)
    assert calc.evaluate_decision(500.0, 3.00) == "Self-Hosted"

def test_zero_mlops_cost_break_even():
    calc = TCOCalculator(gpu_hourly_cost=10.0, mlops_monthly_cost=0.0)
    # Fixed = $7,200/mo. API = $2.00/M. Monthly tokens = 3.6B -> Daily = 120M
    be = calc.calculate_break_even(2.00)
    assert be == pytest.approx(120000000.0, rel=1e-3)

Troubleshooting

Troubleshooting: Day 242 - Open Weights versus Closed APIs

Common Issues

  1. Omission of MLOps overhead:
    • Cause: Computing GPU hardware cost without engineering maintenance.
    • Fix: Include operational engineering overhead in fixed monthly cost.

Security notes

Security & Privacy: Day 242 - Open Weights versus Closed APIs

Security Guidance

  • All TCO modeling calculations execute locally in memory.