AI Engineering: Agents and Applications › Full-Stack AI Applications › Day 321
Hands-on lab — Day 321: Vendor Abstraction and Fallbacks
- ← Back to the Day 321 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/ai-engineering/day-321-vendor-abstraction-and-fallbacks/
Commands
Setup
pip install -r requirements/requirements.txt Run
python examples/vendor_router.py Test
bash tests/run_tests.sh File tree
examples/vendor_router.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/vendor_router.py tests/run_tests.sh tests/test_vendor_router.py troubleshooting.md
Lab README
Day 321 Lab: Vendor Abstraction and Fallbacks
Lesson
- Lesson title: Vendor Abstraction and Fallbacks
- Day number: 321 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-321-vendor-abstraction-and-fallbacks
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-321-vendor-abstraction-and-fallbackswhen the site is running.
Purpose
Build a Vendor Router and Fallback Engine in Python supporting multi-provider fallback priority lists, health tracking, and standardized response schemas.
Learning objectives
- Implement vendor-neutral model invocation interfaces.
- Manage fallback priority lists across primary, secondary, and tertiary providers.
- Intercept simulated upstream errors and execute sub-second failover.
- Track attempted providers and failover status.
Prerequisites
- Python 3.10+ installed
- pytest installed
Supported operating systems
- macOS, Linux, Windows WSL2
Hardware requirements
- Standard CPU, 512MB RAM
Required software
- Python 3.10+, pytest
Free and open-source options
- Python Standard Library, Pytest
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/vendor_router.py: Starter implementation skeletonexamples/vendor_router.py: Verified reference implementationtests/test_vendor_router.py: Test suiteexpected-output/: Captured execution logs
How to run
python3 examples/vendor_router.py
What the commands do
- Executes requests across provider fallback cascades.
Expected output
Normal: {'status': 'SUCCESS', 'resolved_provider': 'anthropic', 'provider_name': 'Anthropic Claude 3.5', 'response': '[Anthropic Claude 3.5] Completed: Hello', 'attempted_providers': ['anthropic'], 'fallback_occurred': False}
Validation steps
Run test runner:
bash tests/run_tests.sh
Tests
The test suite validates:
- Successful resolution on primary provider
- Seamless failover to secondary provider when primary fails
- Fallback to tertiary replica when primary and secondary fail
- Reporting when all configured providers are unavailable
- Health state toggling and tracking
Cleanup
rm -rf .pytest_cache __pycache__ .venv
Troubleshooting
Verify fallback list iteration order matches priority configuration.
Security notes
Runs completely offline on local CPU.
Extension exercises
Implement latency-based dynamic provider weight adjustments.
Navigation
Day number: 321 of 365
Expected output
FIELDS.md
# Measured Values
- `providers_registered`: 3
- `failovers_handled`: 3
examples-run.txt
Normal: {'status': 'SUCCESS', 'resolved_provider': 'anthropic', 'provider_name': 'Anthropic Claude 3.5', 'response': '[Anthropic Claude 3.5] Completed: Hello', 'attempted_providers': ['anthropic'], 'fallback_occurred': False}
measured-values.txt
providers_registered=3
failovers_handled=3
run-output.txt
Normal: {'status': 'SUCCESS', 'resolved_provider': 'anthropic', 'provider_name': 'Anthropic Claude 3.5', 'response': '[Anthropic Claude 3.5] Completed: Hello', 'attempted_providers': ['anthropic'], 'fallback_occurred': False}
starter-run.txt
An untouched starter fails the suite. Captured from a real run:
self = <examples.vendor_router.VendorFallbackRouter object at 0x10971a650>
provider_id = 'anthropic', healthy = False
def set_provider_health(self, provider_id: str, healthy: bool):
> raise NotImplementedError('TASK 1: implement set_provider_health.')
E NotImplementedError: TASK 1: implement set_provider_health.
examples/vendor_router.py:18: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_vendor_router.py::test_primary_provider_success - NotImplem...
FAILED tests/test_vendor_router.py::test_secondary_failover_when_primary_fails
FAILED tests/test_vendor_router.py::test_tertiary_fallback_when_top_two_fail
FAILED tests/test_vendor_router.py::test_all_providers_failed - NotImplemente...
FAILED tests/test_vendor_router.py::test_unhealthy_provider_flag - NotImpleme...
============================== 5 failed in 0.03s ===============================
Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.
test-output.txt
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-321-vendor-abstraction-and-fallbacks
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items
tests/test_vendor_router.py::test_primary_provider_success PASSED [ 20%]
tests/test_vendor_router.py::test_secondary_failover_when_primary_fails PASSED [ 40%]
tests/test_vendor_router.py::test_tertiary_fallback_when_top_two_fail PASSED [ 60%]
tests/test_vendor_router.py::test_all_providers_failed PASSED [ 80%]
tests/test_vendor_router.py::test_unhealthy_provider_flag PASSED [100%]
============================== 5 passed in 0.01s ===============================
test-run.txt
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-321-vendor-abstraction-and-fallbacks
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items
tests/test_vendor_router.py::test_primary_provider_success PASSED [ 20%]
tests/test_vendor_router.py::test_secondary_failover_when_primary_fails PASSED [ 40%]
tests/test_vendor_router.py::test_tertiary_fallback_when_top_two_fail PASSED [ 60%]
tests/test_vendor_router.py::test_all_providers_failed PASSED [ 80%]
tests/test_vendor_router.py::test_unhealthy_provider_flag PASSED [100%]
============================== 5 passed in 0.01s ===============================
Source files
examples/vendor_router.py (2041 bytes)
import time
from typing import Dict, Any, List, Optional
class VendorFallbackRouter:
def __init__(self):
self.providers: Dict[str, Dict[str, Any]] = {
"anthropic": {"name": "Anthropic Claude 3.5", "healthy": True, "consecutive_fails": 0},
"openai": {"name": "OpenAI GPT-4o", "healthy": True, "consecutive_fails": 0},
"vllm_local": {"name": "Self-Hosted Llama 3.3", "healthy": True, "consecutive_fails": 0}
}
self.priority_order = ["anthropic", "openai", "vllm_local"]
def set_provider_health(self, provider_id: str, healthy: bool):
if provider_id in self.providers:
self.providers[provider_id]["healthy"] = bool(healthy)
def call_model_with_fallback(self, prompt: str, simulated_failing_providers: Optional[List[str]] = None) -> Dict[str, Any]:
failing = set(simulated_failing_providers or [])
attempted_providers = []
for provider_id in self.priority_order:
provider = self.providers[provider_id]
attempted_providers.append(provider_id)
if not provider["healthy"] or provider_id in failing:
provider["consecutive_fails"] += 1
continue
provider["consecutive_fails"] = 0
return {
"status": "SUCCESS",
"resolved_provider": provider_id,
"provider_name": provider["name"],
"response": f"[{provider['name']}] Completed: {prompt}",
"attempted_providers": attempted_providers,
"fallback_occurred": len(attempted_providers) > 1
}
return {
"status": "ALL_PROVIDERS_FAILED",
"attempted_providers": attempted_providers,
"error": "503 All upstream model providers are unavailable"
}
if __name__ == "__main__":
router = VendorFallbackRouter()
print("Normal:", router.call_model_with_fallback("Hello"))
metadata.yml (400 bytes)
lesson_id: D321
day: 321
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/vendor_router.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/vendor_router.py (1244 bytes)
"""STARTER -- implement each TASK below, then run the tests.
Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""
import time
from typing import Dict, Any, List, Optional
class VendorFallbackRouter:
def __init__(self):
self.providers: Dict[str, Dict[str, Any]] = {'anthropic': {'name': 'Anthropic Claude 3.5', 'healthy': True, 'consecutive_fails': 0}, 'openai': {'name': 'OpenAI GPT-4o', 'healthy': True, 'consecutive_fails': 0}, 'vllm_local': {'name': 'Self-Hosted Llama 3.3', 'healthy': True, 'consecutive_fails': 0}}
self.priority_order = ['anthropic', 'openai', 'vllm_local']
def set_provider_health(self, provider_id: str, healthy: bool):
raise NotImplementedError('TASK 1: implement set_provider_health.')
def call_model_with_fallback(self, prompt: str, simulated_failing_providers: Optional[List[str]]=None) -> Dict[str, Any]:
raise NotImplementedError('TASK 2: implement call_model_with_fallback.')
if __name__ == '__main__':
router = VendorFallbackRouter()
print('Normal:', router.call_model_with_fallback('Hello'))
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
$PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
pytest tests/test_*.py -v
else
python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_vendor_router.py (1743 bytes)
import pytest
from examples.vendor_router import VendorFallbackRouter
def test_primary_provider_success():
router = VendorFallbackRouter()
res = router.call_model_with_fallback("Write a poem")
assert res["status"] == "SUCCESS"
assert res["resolved_provider"] == "anthropic"
assert res["fallback_occurred"] is False
assert res["attempted_providers"] == ["anthropic"]
def test_secondary_failover_when_primary_fails():
router = VendorFallbackRouter()
res = router.call_model_with_fallback("Write a poem", simulated_failing_providers=["anthropic"])
assert res["status"] == "SUCCESS"
assert res["resolved_provider"] == "openai"
assert res["fallback_occurred"] is True
assert res["attempted_providers"] == ["anthropic", "openai"]
def test_tertiary_fallback_when_top_two_fail():
router = VendorFallbackRouter()
res = router.call_model_with_fallback("Write a poem", simulated_failing_providers=["anthropic", "openai"])
assert res["status"] == "SUCCESS"
assert res["resolved_provider"] == "vllm_local"
assert res["fallback_occurred"] is True
assert len(res["attempted_providers"]) == 3
def test_all_providers_failed():
router = VendorFallbackRouter()
res = router.call_model_with_fallback("Write a poem", simulated_failing_providers=["anthropic", "openai", "vllm_local"])
assert res["status"] == "ALL_PROVIDERS_FAILED"
assert "503" in res["error"]
def test_unhealthy_provider_flag():
router = VendorFallbackRouter()
router.set_provider_health("anthropic", False)
res = router.call_model_with_fallback("Write a poem")
assert res["status"] == "SUCCESS"
assert res["resolved_provider"] == "openai"
assert res["fallback_occurred"] is True