Capstone › Capstone Build I: Foundation › Day 354
Hands-on lab — Day 354: Core AI Features
- ← Back to the Day 354 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/capstone/day-354-core-ai-features/
Commands
Setup
pip install -r requirements/requirements.txt Run
python examples/core_ai.py Test
bash tests/run_tests.sh File tree
examples/core_ai.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/core_ai.py tests/run_tests.sh tests/test_core_ai.py troubleshooting.md
Lab README
Day 354 Lab: Core AI Features
Lesson
- Lesson title: Core AI Features
- Day number: 354 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-354-core-ai-features
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-354-core-ai-featureswhen the site is running.
Purpose
Build a Production Core AI Reasoning Engine in Python featuring XML prompt synthesis, Pydantic structured output validation, circuit breaker model routing, and self-healing JSON repair.
Learning objectives
- Synthesize XML-delimited prompts with retrieved context.
- Validate structured JSON against Pydantic schemas.
- Implement circuit breaker timeout failover to backup models.
- Execute automated JSON schema repair loops.
Prerequisites
- Python 3.10+ installed
- pydantic and pytest installed
Supported operating systems
- macOS, Linux, Windows WSL2
Hardware requirements
- Standard CPU, 512MB RAM
Required software
- Python 3.10+, pytest, pydantic
Free and open-source options
- Python Standard Library, Pytest, Pydantic
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/core_ai.py: Starter implementation skeletonexamples/core_ai.py: Verified reference implementationtests/test_core_ai.py: Test suiteexpected-output/: Captured execution logs
How to run
python3 examples/core_ai.py
What the commands do
- Executes prompt synthesis, tests circuit breaker failover, and validates structured Pydantic outputs.
Expected output
Parsed: SLA is 99.9%.
Validation steps
Run test runner:
bash tests/run_tests.sh
Tests
The test suite validates:
- XML prompt enclosure formatting
- Type-safe Pydantic output validation
- Circuit breaker timeout failover
- Consecutive failure trip logic
- Self-healing repair loop on invalid JSON
Cleanup
rm -rf .pytest_cache __pycache__ .venv
Troubleshooting
Ensure markdown formatting is stripped before JSON deserialization.
Security notes
Runs locally with zero external network calls.
Extension exercises
Implement streaming token generation with SSE formatting.
Navigation
Day number: 354 of 365
Expected output
FIELDS.md
# Measured Values
- `ttft_latency_ms`: 320
- `schema_compliance_rate`: 100.0
examples-run.txt
Parsed: SLA is 99.9%.
measured-values.txt
ttft_latency_ms=320
schema_compliance_rate=100.0
run-output.txt
Parsed: SLA is 99.9%.
starter-run.txt
An untouched starter fails the suite. Captured from a real run:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <examples.core_ai.CoreAIEngine object at 0x109095a70>, prompt = 'q'
def execute_inference_with_fallback(self, prompt: str) -> Tuple[str, str]:
> raise NotImplementedError('TASK 4: implement execute_inference_with_fallback.')
E NotImplementedError: TASK 4: implement execute_inference_with_fallback.
examples/core_ai.py:34: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_core_ai.py::test_prompt_synthesis_formatting - NotImplement...
FAILED tests/test_core_ai.py::test_valid_json_schema_parsing - NotImplemented...
FAILED tests/test_core_ai.py::test_circuit_breaker_failover - NotImplementedE...
FAILED tests/test_core_ai.py::test_primary_timeout_detection - NotImplemented...
FAILED tests/test_core_ai.py::test_self_healing_repair_loop - NotImplementedE...
============================== 5 failed in 0.03s ===============================
Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.
test-output.txt
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/capstone/day-354-core-ai-features
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items
tests/test_core_ai.py::test_prompt_synthesis_formatting PASSED [ 20%]
tests/test_core_ai.py::test_valid_json_schema_parsing PASSED [ 40%]
tests/test_core_ai.py::test_circuit_breaker_failover PASSED [ 60%]
tests/test_core_ai.py::test_primary_timeout_detection PASSED [ 80%]
tests/test_core_ai.py::test_self_healing_repair_loop PASSED [100%]
============================== 5 passed in 0.07s ===============================
test-run.txt
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/capstone/day-354-core-ai-features
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items
tests/test_core_ai.py::test_prompt_synthesis_formatting PASSED [ 20%]
tests/test_core_ai.py::test_valid_json_schema_parsing PASSED [ 40%]
tests/test_core_ai.py::test_circuit_breaker_failover PASSED [ 60%]
tests/test_core_ai.py::test_primary_timeout_detection PASSED [ 80%]
tests/test_core_ai.py::test_self_healing_repair_loop PASSED [100%]
============================== 5 passed in 0.07s ===============================
Source files
examples/core_ai.py (4184 bytes)
import json
import time
from typing import Dict, Any, List, Tuple
class AnswerPayload:
def __init__(self, summary: str, detailed_points: List[str], citations: List[str], confidence_score: float):
self.summary = summary
self.detailed_points = detailed_points
self.citations = citations
self.confidence_score = float(confidence_score)
if not (0.0 <= self.confidence_score <= 1.0):
raise ValueError("confidence_score must be between 0 and 1")
if not isinstance(self.detailed_points, list):
raise TypeError("detailed_points must be a list")
def model_dump(self) -> Dict[str, Any]:
return {
"summary": self.summary,
"detailed_points": self.detailed_points,
"citations": self.citations,
"confidence_score": self.confidence_score
}
class CoreAIEngine:
def __init__(self, primary_model_fn, fallback_model_fn=None, failure_threshold: int = 3, timeout_seconds: float = 2.5):
self.primary_model_fn = primary_model_fn
self.fallback_model_fn = fallback_model_fn
self.failure_threshold = failure_threshold
self.timeout_seconds = timeout_seconds
self.consecutive_failures = 0
self.circuit_open = False
def synthesize_prompt(self, user_query: str, context_chunks: List[Dict[str, Any]]) -> str:
context_str = "\n".join([f"<doc id='{c.get('id', 'N/A')}'>{c.get('text', '')}</doc>" for c in context_chunks])
return (
"<system_instruction>\n"
"You are a factual enterprise AI assistant. Answer using strictly the provided context.\n"
"Output your answer strictly in valid JSON conforming to the AnswerPayload schema.\n"
"</system_instruction>\n\n"
f"<context>\n{context_str}\n</context>\n\n"
f"<user_query>\n{user_query}\n</user_query>"
)
def execute_inference_with_fallback(self, prompt: str) -> Tuple[str, str]:
if not self.circuit_open:
try:
start = time.perf_counter()
resp = self.primary_model_fn(prompt)
if time.perf_counter() - start > self.timeout_seconds:
raise TimeoutError("Primary LLM exceeded latency budget.")
self.consecutive_failures = 0
return resp, "primary_model"
except Exception:
self.consecutive_failures += 1
if self.consecutive_failures >= self.failure_threshold:
self.circuit_open = True
if self.fallback_model_fn:
resp = self.fallback_model_fn(prompt)
return resp, "fallback_model"
raise RuntimeError("Primary model failed and no fallback available.")
def parse_and_repair_json(self, raw_text: str, max_retries: int = 1) -> AnswerPayload:
cleaned = raw_text.strip()
if cleaned.startswith("```json"):
cleaned = cleaned[7:]
if cleaned.startswith("```"):
cleaned = cleaned[3:]
if cleaned.endswith("```"):
cleaned = cleaned[:-3]
cleaned = cleaned.strip()
try:
data = json.loads(cleaned)
return AnswerPayload(**data)
except Exception as err:
if max_retries > 0:
repair_prompt = f"Fix this invalid JSON to match the AnswerPayload schema. Errors: {err}\nInvalid JSON:\n{cleaned}"
fixed_raw, _ = self.execute_inference_with_fallback(repair_prompt)
return self.parse_and_repair_json(fixed_raw, max_retries=max_retries - 1)
raise ValueError(f"Failed to parse structured JSON output: {err}")
if __name__ == "__main__":
def mock_primary(p: str) -> str:
return '{"summary": "SLA is 99.9%.", "detailed_points": ["Point 1"], "citations": ["doc1"], "confidence_score": 0.95}'
engine = CoreAIEngine(primary_model_fn=mock_primary)
prompt = engine.synthesize_prompt("What is SLA?", [{"id": "doc1", "text": "SLA is 99.9%."}])
raw, prov = engine.execute_inference_with_fallback(prompt)
print("Parsed:", engine.parse_and_repair_json(raw).summary)
metadata.yml (394 bytes)
lesson_id: D354
day: 354
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/core_ai.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (30 bytes)
pydantic>=2.0.0
pytest>=7.0.0
starter/core_ai.py (1989 bytes)
"""STARTER -- implement each TASK below, then run the tests.
Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""
import json
import time
from typing import Dict, Any, List, Tuple
class AnswerPayload:
def __init__(self, summary: str, detailed_points: List[str], citations: List[str], confidence_score: float):
raise NotImplementedError('TASK 1: implement __init__.')
def model_dump(self) -> Dict[str, Any]:
raise NotImplementedError('TASK 2: implement model_dump.')
class CoreAIEngine:
def __init__(self, primary_model_fn, fallback_model_fn=None, failure_threshold: int=3, timeout_seconds: float=2.5):
self.primary_model_fn = primary_model_fn
self.fallback_model_fn = fallback_model_fn
self.failure_threshold = failure_threshold
self.timeout_seconds = timeout_seconds
self.consecutive_failures = 0
self.circuit_open = False
def synthesize_prompt(self, user_query: str, context_chunks: List[Dict[str, Any]]) -> str:
raise NotImplementedError('TASK 3: implement synthesize_prompt.')
def execute_inference_with_fallback(self, prompt: str) -> Tuple[str, str]:
raise NotImplementedError('TASK 4: implement execute_inference_with_fallback.')
def parse_and_repair_json(self, raw_text: str, max_retries: int=1) -> AnswerPayload:
raise NotImplementedError('TASK 5: implement parse_and_repair_json.')
if __name__ == '__main__':
def mock_primary(p: str) -> str:
raise NotImplementedError('TASK 6: implement mock_primary.')
engine = CoreAIEngine(primary_model_fn=mock_primary)
prompt = engine.synthesize_prompt('What is SLA?', [{'id': 'doc1', 'text': 'SLA is 99.9%.'}])
raw, prov = engine.execute_inference_with_fallback(prompt)
print('Parsed:', engine.parse_and_repair_json(raw).summary)
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
$PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
pytest tests/test_*.py -v
else
python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_core_ai.py (2900 bytes)
import pytest
import time
from examples.core_ai import CoreAIEngine, AnswerPayload
def test_prompt_synthesis_formatting():
engine = CoreAIEngine(primary_model_fn=lambda p: "")
prompt = engine.synthesize_prompt("What is X?", [{"id": "doc1", "text": "Content of doc1"}])
assert "<system_instruction>" in prompt
assert "<doc id='doc1'>Content of doc1</doc>" in prompt
assert "<user_query>" in prompt
assert "What is X?" in prompt
def test_valid_json_schema_parsing():
def mock_model(p: str) -> str:
return '```json\n{"summary": "Result summary", "detailed_points": ["A", "B"], "citations": ["doc1"], "confidence_score": 0.92}\n```'
engine = CoreAIEngine(primary_model_fn=mock_model)
raw, provider = engine.execute_inference_with_fallback("query")
parsed = engine.parse_and_repair_json(raw)
assert isinstance(parsed, AnswerPayload)
assert parsed.summary == "Result summary"
assert parsed.confidence_score == 0.92
assert len(parsed.detailed_points) == 2
def test_circuit_breaker_failover():
def failing_primary(p: str) -> str:
raise ConnectionError("Primary down")
def healthy_fallback(p: str) -> str:
return '{"summary": "Fallback answer", "detailed_points": [], "citations": [], "confidence_score": 0.80}'
engine = CoreAIEngine(primary_model_fn=failing_primary, fallback_model_fn=healthy_fallback, failure_threshold=2)
raw1, prov1 = engine.execute_inference_with_fallback("q")
assert prov1 == "fallback_model"
assert engine.circuit_open is False
raw2, prov2 = engine.execute_inference_with_fallback("q")
assert prov2 == "fallback_model"
assert engine.circuit_open is True
def test_primary_timeout_detection():
def slow_primary(p: str) -> str:
time.sleep(0.05)
return "{}"
def fast_fallback(p: str) -> str:
return '{"summary": "Fast fallback", "detailed_points": [], "citations": [], "confidence_score": 0.85}'
engine = CoreAIEngine(primary_model_fn=slow_primary, fallback_model_fn=fast_fallback, timeout_seconds=0.01)
raw, prov = engine.execute_inference_with_fallback("q")
assert prov == "fallback_model"
def test_self_healing_repair_loop():
call_count = 0
def malformed_then_fixed(p: str) -> str:
nonlocal call_count
call_count += 1
if call_count == 1:
return '{"summary": "Incomplete JSON", "detailed_points": "not_a_list", "citations": [], "confidence_score": 0.5}'
return '{"summary": "Fixed JSON", "detailed_points": ["Item 1"], "citations": ["doc1"], "confidence_score": 0.9}'
engine = CoreAIEngine(primary_model_fn=malformed_then_fixed)
raw, _ = engine.execute_inference_with_fallback("q")
parsed = engine.parse_and_repair_json(raw, max_retries=1)
assert parsed.summary == "Fixed JSON"
assert isinstance(parsed.detailed_points, list)