LLMs and Generative AI › The LLM Landscape › Day 242
Hands-on lab — Day 242: Open Weights versus Closed APIs
- ← Back to the Day 242 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/llms-and-generative-ai/day-242-open-weights-versus-closed-apis/
Commands
Setup
pip install -r requirements/requirements.txt Run
python3 examples/open_weights_versus_closed_apis_lib.py Test
./tests/run_tests.sh File tree
conftest.py examples/open_weights_versus_closed_apis_lib.py examples/test_open_weights_versus_closed_apis_lib.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/starter-run.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/open_weights_versus_closed_apis_lib.py starter/test_open_weights_versus_closed_apis_lib.py tests/run_tests.sh tests/test_open_weights_versus_closed_apis_lib.py troubleshooting.md
Lab README
Lab: Day 242 -- Open Weights versus Closed APIs
Lesson
Day number: 242 of 365. Course: Course06-SS01 (LLMs and Generative AI - Working with LLMs). Topic: Open Weights vs. Closed APIs, Total Cost of Ownership (TCO), and vLLM Serving.
Purpose
Build and test an automated Total Cost of Ownership (TCO) and inference throughput simulator in Python. Calculate monthly break-even thresholds, evaluate high-volume vs low-volume enterprise deployment scenarios, and determine the optimal infrastructure choice.
Learning objectives
- Formulate the TCO break-even equation comparing GPU server rental against API token charges.
- Evaluate deployment decisions based on daily token volume thresholds.
- Understand the operational role of vLLM PagedAttention and continuous batching.
- Assess data sovereignty and regulatory compliance constraints.
Prerequisites
- Day 241 (The Model Landscape: Claude, GPT, Gemini, Llama).
- Python 3.11+ with PyTorch and NumPy.
Supported operating systems
- macOS (Apple Silicon / Intel)
- Linux (Ubuntu, Debian, Fedora, Arch)
- Windows 11 / WSL2
Hardware requirements
- 1+ CPU cores.
- 1 GB RAM.
- 100 MB disk space.
Required software
- Python 3.11 or newer.
- pip package manager.
- virtualenv or venv module.
Free and open-source options
Python and standard math libraries are open source.
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/open_weights_versus_closed_apis_lib.py: Student scaffold file.examples/open_weights_versus_closed_apis_lib.py: Complete reference implementation.tests/test_open_weights_versus_closed_apis_lib.py: Pytest automated validation suite.expected-output/: Verified output logs and baseline values.
How to run
Execute the reference demonstration script:
python3 examples/open_weights_versus_closed_apis_lib.py
What the commands do
- Computes daily break-even token thresholds.
- Evaluates deployment recommendations across volume scenarios.
- Runs unit test assertions.
Expected output
TCO Demo: Break-Even = 247.56M Tok/Day, Low Vol = Closed API, High Vol = Self-Hosted
Validation steps
- Verify
calculate_break_evencomputes exact break-even daily tokens. - Confirm
evaluate_decisionrecommends Closed API for volumes below break-even. - Confirm
evaluate_decisionrecommends Self-Hosted for volumes above break-even. - Ensure all unit test assertions pass.
Tests
Run the test runner script:
./tests/run_tests.sh
Cleanup
find . -type d -name "__pycache__" -exec rm -rf {} +
find . -type d -name ".pytest_cache" -exec rm -rf {} +
Troubleshooting
- Break-even scaling error: Ensure monthly token totals are converted to daily averages by dividing by 30.
Security notes
All TCO simulations execute locally in process memory.
Extension exercises
- Implement an automated GPU cluster reservation amortization tool.
- Build a latency simulator measuring tokens/second under varying batch sizes.
Navigation
- Lesson title: Open Weights versus Closed APIs
- Day number: 242 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-242-open-weights-versus-closed-apis
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-242-open-weights-versus-closed-apiswhen the site is running.
Expected output
FIELDS.md
# Expected Output Fields: Day 242
- `Break-Even`: Daily token volume threshold where self-hosting breaks even with closed APIs.
- `Low Vol`: Infrastructure recommendation for low query traffic.
- `High Vol`: Infrastructure recommendation for high query traffic.
examples-run.txt
TCO Demo: Break-Even = 247.56M Tok/Day, Low Vol = Closed API, High Vol = Self-Hosted
measured-values.txt
Break-Even: 247.56M Tok/Day
Low Vol: Closed API
High Vol: Self-Hosted
starter-run.txt
Starter scaffold executed. Ready for student implementation.
test-run.txt
============================= test session starts ==============================
collected 3 items
tests/test_open_weights_versus_closed_apis_lib.py::test_break_even_calculation PASSED [ 33%]
tests/test_open_weights_versus_closed_apis_lib.py::test_decision_evaluation PASSED [ 66%]
tests/test_open_weights_versus_closed_apis_lib.py::test_zero_mlops_cost_break_even PASSED [100%]
============================== 3 passed in 0.40s ===============================
Source files
conftest.py (585 bytes)
"""Make this lab importable when the suite runs.
pytest puts the test file's own directory on sys.path, not the lab root, so a
test that does `from examples.<module> import ...` -- or imports a module that
lives in examples/ directly -- fails without this. Putting it here rather than
in run_tests.sh means a bare `pytest` from the lab root works too.
"""
import os
import sys
LAB = os.path.dirname(os.path.abspath(__file__))
for candidate in (LAB, os.path.join(LAB, "examples")):
if os.path.isdir(candidate) and candidate not in sys.path:
sys.path.insert(0, candidate)
examples/open_weights_versus_closed_apis_lib.py (1433 bytes)
from typing import Dict, Any
class TCOCalculator:
def __init__(self, gpu_hourly_cost: float = 24.0, mlops_monthly_cost: float = 5000.0):
self.gpu_hourly_cost = gpu_hourly_cost
self.mlops_monthly_cost = mlops_monthly_cost
def calculate_break_even(self, api_rate_per_million: float) -> float:
fixed_monthly = (self.gpu_hourly_cost * 24 * 30) + self.mlops_monthly_cost
monthly_tokens = (fixed_monthly / api_rate_per_million) * 1e6
daily_tokens = monthly_tokens / 30.0
return round(daily_tokens, 2)
def evaluate_decision(self, daily_tokens_millions: float, api_rate_per_million: float) -> str:
monthly_tokens = daily_tokens_millions * 1e6 * 30
api_monthly = (monthly_tokens / 1e6) * api_rate_per_million
self_host_monthly = (self.gpu_hourly_cost * 24 * 30) + self.mlops_monthly_cost
return "Self-Hosted" if self_host_monthly < api_monthly else "Closed API"
def run_tco_demo():
calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
be_daily = calc.calculate_break_even(3.00) # $3/M API
dec_low = calc.evaluate_decision(10.0, 3.00) # 10M tokens/day
dec_high = calc.evaluate_decision(500.0, 3.00) # 500M tokens/day
print(f"TCO Demo: Break-Even = {be_daily/1e6:.2f}M Tok/Day, Low Vol = {dec_low}, High Vol = {dec_high}")
return be_daily, dec_low, dec_high
if __name__ == "__main__":
run_tco_demo()
examples/test_open_weights_versus_closed_apis_lib.py (1139 bytes)
import pytest
from examples.open_weights_versus_closed_apis_lib import TCOCalculator
def test_break_even_calculation():
# 8x H100 node: $24/hr * 720 = $17,280 + $5,000 MLOps = $22,280 / mo
# API: $3.00 / M -> $22,280 / 0.000003 = 7,426,666,667 tokens / mo
# Daily: ~247,555,555 tokens / day (~247.56M)
calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
be = calc.calculate_break_even(3.00)
assert be == pytest.approx(247555555.56, rel=1e-3)
def test_decision_evaluation():
calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
# 10M daily -> API is vastly cheaper ($900 vs $22,280)
assert calc.evaluate_decision(10.0, 3.00) == "Closed API"
# 500M daily -> Self-hosting is vastly cheaper ($45,000 vs $22,280)
assert calc.evaluate_decision(500.0, 3.00) == "Self-Hosted"
def test_zero_mlops_cost_break_even():
calc = TCOCalculator(gpu_hourly_cost=10.0, mlops_monthly_cost=0.0)
# Fixed = $7,200/mo. API = $2.00/M. Monthly tokens = 3.6B -> Daily = 120M
be = calc.calculate_break_even(2.00)
assert be == pytest.approx(120000000.0, rel=1e-3)
metadata.yml (444 bytes)
lesson_id: D242
day: 242
kind: lab
languages:
- python
setup_commands:
- 'pip install -r requirements/requirements.txt'
run_commands:
- 'python3 examples/open_weights_versus_closed_apis_lib.py'
test_commands:
- './tests/run_tests.sh'
cleanup_commands:
- 'find . -type d -name "__pycache__" -exec rm -rf {} +'
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-29'
executed_on: 'macos-arm64'
requirements/requirements.txt (41 bytes)
torch>=2.2.0
numpy>=1.26.0
pytest>=8.0.0
starter/open_weights_versus_closed_apis_lib.py (552 bytes)
from typing import Dict, Any
class TCOCalculator:
def __init__(self, gpu_hourly_cost: float = 24.0, mlops_monthly_cost: float = 5000.0):
self.gpu_hourly_cost = gpu_hourly_cost
self.mlops_monthly_cost = mlops_monthly_cost
def calculate_break_even(self, api_rate_per_million: float) -> float:
# TODO: Calculate break-even daily token volume
pass
def evaluate_decision(self, daily_tokens_millions: float, api_rate_per_million: float) -> str:
# TODO: Return "Closed API" or "Self-Hosted"
pass
starter/test_open_weights_versus_closed_apis_lib.py (1139 bytes)
import pytest
from examples.open_weights_versus_closed_apis_lib import TCOCalculator
def test_break_even_calculation():
# 8x H100 node: $24/hr * 720 = $17,280 + $5,000 MLOps = $22,280 / mo
# API: $3.00 / M -> $22,280 / 0.000003 = 7,426,666,667 tokens / mo
# Daily: ~247,555,555 tokens / day (~247.56M)
calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
be = calc.calculate_break_even(3.00)
assert be == pytest.approx(247555555.56, rel=1e-3)
def test_decision_evaluation():
calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
# 10M daily -> API is vastly cheaper ($900 vs $22,280)
assert calc.evaluate_decision(10.0, 3.00) == "Closed API"
# 500M daily -> Self-hosting is vastly cheaper ($45,000 vs $22,280)
assert calc.evaluate_decision(500.0, 3.00) == "Self-Hosted"
def test_zero_mlops_cost_break_even():
calc = TCOCalculator(gpu_hourly_cost=10.0, mlops_monthly_cost=0.0)
# Fixed = $7,200/mo. API = $2.00/M. Monthly tokens = 3.6B -> Daily = 120M
be = calc.calculate_break_even(2.00)
assert be == pytest.approx(120000000.0, rel=1e-3)
tests/run_tests.sh (227 bytes)
#!/usr/bin/env bash
set -euo pipefail
echo "========================================"
echo "Running Day 242 Lab Test Suite"
echo "========================================"
pytest tests/ -v
echo "All tests passed successfully."
tests/test_open_weights_versus_closed_apis_lib.py (1139 bytes)
import pytest
from examples.open_weights_versus_closed_apis_lib import TCOCalculator
def test_break_even_calculation():
# 8x H100 node: $24/hr * 720 = $17,280 + $5,000 MLOps = $22,280 / mo
# API: $3.00 / M -> $22,280 / 0.000003 = 7,426,666,667 tokens / mo
# Daily: ~247,555,555 tokens / day (~247.56M)
calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
be = calc.calculate_break_even(3.00)
assert be == pytest.approx(247555555.56, rel=1e-3)
def test_decision_evaluation():
calc = TCOCalculator(gpu_hourly_cost=24.0, mlops_monthly_cost=5000.0)
# 10M daily -> API is vastly cheaper ($900 vs $22,280)
assert calc.evaluate_decision(10.0, 3.00) == "Closed API"
# 500M daily -> Self-hosting is vastly cheaper ($45,000 vs $22,280)
assert calc.evaluate_decision(500.0, 3.00) == "Self-Hosted"
def test_zero_mlops_cost_break_even():
calc = TCOCalculator(gpu_hourly_cost=10.0, mlops_monthly_cost=0.0)
# Fixed = $7,200/mo. API = $2.00/M. Monthly tokens = 3.6B -> Daily = 120M
be = calc.calculate_break_even(2.00)
assert be == pytest.approx(120000000.0, rel=1e-3)
Troubleshooting
Troubleshooting: Day 242 - Open Weights versus Closed APIs
Common Issues
- Omission of MLOps overhead:
- Cause: Computing GPU hardware cost without engineering maintenance.
- Fix: Include operational engineering overhead in fixed monthly cost.
Security notes
Security & Privacy: Day 242 - Open Weights versus Closed APIs
Security Guidance
- All TCO modeling calculations execute locally in memory.