AI Engineering: Agents and Applications › Full-Stack AI Applications › Day 321

Hands-on lab — Day 321: Vendor Abstraction and Fallbacks

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/vendor_router.py

Test

bash tests/run_tests.sh

File tree

examples/vendor_router.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/vendor_router.py
tests/run_tests.sh
tests/test_vendor_router.py
troubleshooting.md

Lab README

Day 321 Lab: Vendor Abstraction and Fallbacks

Lesson

Purpose

Build a Vendor Router and Fallback Engine in Python supporting multi-provider fallback priority lists, health tracking, and standardized response schemas.

Learning objectives

  • Implement vendor-neutral model invocation interfaces.
  • Manage fallback priority lists across primary, secondary, and tertiary providers.
  • Intercept simulated upstream errors and execute sub-second failover.
  • Track attempted providers and failover status.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library, Pytest

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/vendor_router.py: Starter implementation skeleton
  • examples/vendor_router.py: Verified reference implementation
  • tests/test_vendor_router.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/vendor_router.py

What the commands do

  • Executes requests across provider fallback cascades.

Expected output

Normal: {'status': 'SUCCESS', 'resolved_provider': 'anthropic', 'provider_name': 'Anthropic Claude 3.5', 'response': '[Anthropic Claude 3.5] Completed: Hello', 'attempted_providers': ['anthropic'], 'fallback_occurred': False}

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Successful resolution on primary provider
  • Seamless failover to secondary provider when primary fails
  • Fallback to tertiary replica when primary and secondary fail
  • Reporting when all configured providers are unavailable
  • Health state toggling and tracking

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Verify fallback list iteration order matches priority configuration.

Security notes

Runs completely offline on local CPU.

Extension exercises

Implement latency-based dynamic provider weight adjustments.

Day number: 321 of 365

Expected output

FIELDS.md

# Measured Values
- `providers_registered`: 3
- `failovers_handled`: 3

examples-run.txt

Normal: {'status': 'SUCCESS', 'resolved_provider': 'anthropic', 'provider_name': 'Anthropic Claude 3.5', 'response': '[Anthropic Claude 3.5] Completed: Hello', 'attempted_providers': ['anthropic'], 'fallback_occurred': False}

measured-values.txt

providers_registered=3
failovers_handled=3

run-output.txt

Normal: {'status': 'SUCCESS', 'resolved_provider': 'anthropic', 'provider_name': 'Anthropic Claude 3.5', 'response': '[Anthropic Claude 3.5] Completed: Hello', 'attempted_providers': ['anthropic'], 'fallback_occurred': False}

starter-run.txt

An untouched starter fails the suite. Captured from a real run:


self = <examples.vendor_router.VendorFallbackRouter object at 0x10971a650>
provider_id = 'anthropic', healthy = False

    def set_provider_health(self, provider_id: str, healthy: bool):
>       raise NotImplementedError('TASK 1: implement set_provider_health.')
E       NotImplementedError: TASK 1: implement set_provider_health.

examples/vendor_router.py:18: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_vendor_router.py::test_primary_provider_success - NotImplem...
FAILED tests/test_vendor_router.py::test_secondary_failover_when_primary_fails
FAILED tests/test_vendor_router.py::test_tertiary_fallback_when_top_two_fail
FAILED tests/test_vendor_router.py::test_all_providers_failed - NotImplemente...
FAILED tests/test_vendor_router.py::test_unhealthy_provider_flag - NotImpleme...
============================== 5 failed in 0.03s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-321-vendor-abstraction-and-fallbacks
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_vendor_router.py::test_primary_provider_success PASSED        [ 20%]
tests/test_vendor_router.py::test_secondary_failover_when_primary_fails PASSED [ 40%]
tests/test_vendor_router.py::test_tertiary_fallback_when_top_two_fail PASSED [ 60%]
tests/test_vendor_router.py::test_all_providers_failed PASSED            [ 80%]
tests/test_vendor_router.py::test_unhealthy_provider_flag PASSED         [100%]

============================== 5 passed in 0.01s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-321-vendor-abstraction-and-fallbacks
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_vendor_router.py::test_primary_provider_success PASSED        [ 20%]
tests/test_vendor_router.py::test_secondary_failover_when_primary_fails PASSED [ 40%]
tests/test_vendor_router.py::test_tertiary_fallback_when_top_two_fail PASSED [ 60%]
tests/test_vendor_router.py::test_all_providers_failed PASSED            [ 80%]
tests/test_vendor_router.py::test_unhealthy_provider_flag PASSED         [100%]

============================== 5 passed in 0.01s ===============================

Source files

examples/vendor_router.py (2041 bytes)
import time
from typing import Dict, Any, List, Optional

class VendorFallbackRouter:
    def __init__(self):
        self.providers: Dict[str, Dict[str, Any]] = {
            "anthropic": {"name": "Anthropic Claude 3.5", "healthy": True, "consecutive_fails": 0},
            "openai": {"name": "OpenAI GPT-4o", "healthy": True, "consecutive_fails": 0},
            "vllm_local": {"name": "Self-Hosted Llama 3.3", "healthy": True, "consecutive_fails": 0}
        }
        self.priority_order = ["anthropic", "openai", "vllm_local"]
        
    def set_provider_health(self, provider_id: str, healthy: bool):
        if provider_id in self.providers:
            self.providers[provider_id]["healthy"] = bool(healthy)
            
    def call_model_with_fallback(self, prompt: str, simulated_failing_providers: Optional[List[str]] = None) -> Dict[str, Any]:
        failing = set(simulated_failing_providers or [])
        attempted_providers = []
        
        for provider_id in self.priority_order:
            provider = self.providers[provider_id]
            attempted_providers.append(provider_id)
            
            if not provider["healthy"] or provider_id in failing:
                provider["consecutive_fails"] += 1
                continue
                
            provider["consecutive_fails"] = 0
            return {
                "status": "SUCCESS",
                "resolved_provider": provider_id,
                "provider_name": provider["name"],
                "response": f"[{provider['name']}] Completed: {prompt}",
                "attempted_providers": attempted_providers,
                "fallback_occurred": len(attempted_providers) > 1
            }
            
        return {
            "status": "ALL_PROVIDERS_FAILED",
            "attempted_providers": attempted_providers,
            "error": "503 All upstream model providers are unavailable"
        }

if __name__ == "__main__":
    router = VendorFallbackRouter()
    print("Normal:", router.call_model_with_fallback("Hello"))
metadata.yml (400 bytes)
lesson_id: D321
day: 321
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/vendor_router.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/vendor_router.py (1244 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

import time
from typing import Dict, Any, List, Optional

class VendorFallbackRouter:

    def __init__(self):
        self.providers: Dict[str, Dict[str, Any]] = {'anthropic': {'name': 'Anthropic Claude 3.5', 'healthy': True, 'consecutive_fails': 0}, 'openai': {'name': 'OpenAI GPT-4o', 'healthy': True, 'consecutive_fails': 0}, 'vllm_local': {'name': 'Self-Hosted Llama 3.3', 'healthy': True, 'consecutive_fails': 0}}
        self.priority_order = ['anthropic', 'openai', 'vllm_local']

    def set_provider_health(self, provider_id: str, healthy: bool):
        raise NotImplementedError('TASK 1: implement set_provider_health.')

    def call_model_with_fallback(self, prompt: str, simulated_failing_providers: Optional[List[str]]=None) -> Dict[str, Any]:
        raise NotImplementedError('TASK 2: implement call_model_with_fallback.')
if __name__ == '__main__':
    router = VendorFallbackRouter()
    print('Normal:', router.call_model_with_fallback('Hello'))
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_vendor_router.py (1743 bytes)
import pytest
from examples.vendor_router import VendorFallbackRouter

def test_primary_provider_success():
    router = VendorFallbackRouter()
    res = router.call_model_with_fallback("Write a poem")
    assert res["status"] == "SUCCESS"
    assert res["resolved_provider"] == "anthropic"
    assert res["fallback_occurred"] is False
    assert res["attempted_providers"] == ["anthropic"]

def test_secondary_failover_when_primary_fails():
    router = VendorFallbackRouter()
    res = router.call_model_with_fallback("Write a poem", simulated_failing_providers=["anthropic"])
    assert res["status"] == "SUCCESS"
    assert res["resolved_provider"] == "openai"
    assert res["fallback_occurred"] is True
    assert res["attempted_providers"] == ["anthropic", "openai"]

def test_tertiary_fallback_when_top_two_fail():
    router = VendorFallbackRouter()
    res = router.call_model_with_fallback("Write a poem", simulated_failing_providers=["anthropic", "openai"])
    assert res["status"] == "SUCCESS"
    assert res["resolved_provider"] == "vllm_local"
    assert res["fallback_occurred"] is True
    assert len(res["attempted_providers"]) == 3

def test_all_providers_failed():
    router = VendorFallbackRouter()
    res = router.call_model_with_fallback("Write a poem", simulated_failing_providers=["anthropic", "openai", "vllm_local"])
    assert res["status"] == "ALL_PROVIDERS_FAILED"
    assert "503" in res["error"]

def test_unhealthy_provider_flag():
    router = VendorFallbackRouter()
    router.set_provider_health("anthropic", False)
    res = router.call_model_with_fallback("Write a poem")
    assert res["status"] == "SUCCESS"
    assert res["resolved_provider"] == "openai"
    assert res["fallback_occurred"] is True

Troubleshooting

Troubleshooting - Day 321\nEnsure priority_order list reflects valid registered provider IDs.\n

Security notes

Security - Day 321\nAll configured fallback providers must satisfy enterprise data privacy and ZDR standards.\n