Capstone › Capstone Build I: Foundation › Day 354

Hands-on lab — Day 354: Core AI Features

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/core_ai.py

Test

bash tests/run_tests.sh

File tree

examples/core_ai.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/core_ai.py
tests/run_tests.sh
tests/test_core_ai.py
troubleshooting.md

Lab README

Day 354 Lab: Core AI Features

Lesson

  • Lesson title: Core AI Features
  • Day number: 354 of 365
  • Lesson article: https://ai-roadmap-365.github.io/day-354-core-ai-features
  • Lab files: everything you need is in this directory — follow “How to run” below.
  • Browse the course locally: from the repository root, this lab also appears in the course website at /labs/day-354-core-ai-features when the site is running.

Purpose

Build a Production Core AI Reasoning Engine in Python featuring XML prompt synthesis, Pydantic structured output validation, circuit breaker model routing, and self-healing JSON repair.

Learning objectives

  • Synthesize XML-delimited prompts with retrieved context.
  • Validate structured JSON against Pydantic schemas.
  • Implement circuit breaker timeout failover to backup models.
  • Execute automated JSON schema repair loops.

Prerequisites

  • Python 3.10+ installed
  • pydantic and pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest, pydantic

Free and open-source options

  • Python Standard Library, Pytest, Pydantic

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/core_ai.py: Starter implementation skeleton
  • examples/core_ai.py: Verified reference implementation
  • tests/test_core_ai.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/core_ai.py

What the commands do

  • Executes prompt synthesis, tests circuit breaker failover, and validates structured Pydantic outputs.

Expected output

Parsed: SLA is 99.9%.

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • XML prompt enclosure formatting
  • Type-safe Pydantic output validation
  • Circuit breaker timeout failover
  • Consecutive failure trip logic
  • Self-healing repair loop on invalid JSON

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Ensure markdown formatting is stripped before JSON deserialization.

Security notes

Runs locally with zero external network calls.

Extension exercises

Implement streaming token generation with SSE formatting.

Day number: 354 of 365

Expected output

FIELDS.md

# Measured Values
- `ttft_latency_ms`: 320
- `schema_compliance_rate`: 100.0

examples-run.txt

Parsed: SLA is 99.9%.

measured-values.txt

ttft_latency_ms=320
schema_compliance_rate=100.0

run-output.txt

Parsed: SLA is 99.9%.

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <examples.core_ai.CoreAIEngine object at 0x109095a70>, prompt = 'q'

    def execute_inference_with_fallback(self, prompt: str) -> Tuple[str, str]:
>       raise NotImplementedError('TASK 4: implement execute_inference_with_fallback.')
E       NotImplementedError: TASK 4: implement execute_inference_with_fallback.

examples/core_ai.py:34: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_core_ai.py::test_prompt_synthesis_formatting - NotImplement...
FAILED tests/test_core_ai.py::test_valid_json_schema_parsing - NotImplemented...
FAILED tests/test_core_ai.py::test_circuit_breaker_failover - NotImplementedE...
FAILED tests/test_core_ai.py::test_primary_timeout_detection - NotImplemented...
FAILED tests/test_core_ai.py::test_self_healing_repair_loop - NotImplementedE...
============================== 5 failed in 0.03s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/capstone/day-354-core-ai-features
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_core_ai.py::test_prompt_synthesis_formatting PASSED           [ 20%]
tests/test_core_ai.py::test_valid_json_schema_parsing PASSED             [ 40%]
tests/test_core_ai.py::test_circuit_breaker_failover PASSED              [ 60%]
tests/test_core_ai.py::test_primary_timeout_detection PASSED             [ 80%]
tests/test_core_ai.py::test_self_healing_repair_loop PASSED              [100%]

============================== 5 passed in 0.07s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/capstone/day-354-core-ai-features
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_core_ai.py::test_prompt_synthesis_formatting PASSED           [ 20%]
tests/test_core_ai.py::test_valid_json_schema_parsing PASSED             [ 40%]
tests/test_core_ai.py::test_circuit_breaker_failover PASSED              [ 60%]
tests/test_core_ai.py::test_primary_timeout_detection PASSED             [ 80%]
tests/test_core_ai.py::test_self_healing_repair_loop PASSED              [100%]

============================== 5 passed in 0.07s ===============================

Source files

examples/core_ai.py (4184 bytes)
import json
import time
from typing import Dict, Any, List, Tuple

class AnswerPayload:
    def __init__(self, summary: str, detailed_points: List[str], citations: List[str], confidence_score: float):
        self.summary = summary
        self.detailed_points = detailed_points
        self.citations = citations
        self.confidence_score = float(confidence_score)
        if not (0.0 <= self.confidence_score <= 1.0):
            raise ValueError("confidence_score must be between 0 and 1")
        if not isinstance(self.detailed_points, list):
            raise TypeError("detailed_points must be a list")

    def model_dump(self) -> Dict[str, Any]:
        return {
            "summary": self.summary,
            "detailed_points": self.detailed_points,
            "citations": self.citations,
            "confidence_score": self.confidence_score
        }

class CoreAIEngine:
    def __init__(self, primary_model_fn, fallback_model_fn=None, failure_threshold: int = 3, timeout_seconds: float = 2.5):
        self.primary_model_fn = primary_model_fn
        self.fallback_model_fn = fallback_model_fn
        self.failure_threshold = failure_threshold
        self.timeout_seconds = timeout_seconds
        self.consecutive_failures = 0
        self.circuit_open = False

    def synthesize_prompt(self, user_query: str, context_chunks: List[Dict[str, Any]]) -> str:
        context_str = "\n".join([f"<doc id='{c.get('id', 'N/A')}'>{c.get('text', '')}</doc>" for c in context_chunks])
        return (
            "<system_instruction>\n"
            "You are a factual enterprise AI assistant. Answer using strictly the provided context.\n"
            "Output your answer strictly in valid JSON conforming to the AnswerPayload schema.\n"
            "</system_instruction>\n\n"
            f"<context>\n{context_str}\n</context>\n\n"
            f"<user_query>\n{user_query}\n</user_query>"
        )

    def execute_inference_with_fallback(self, prompt: str) -> Tuple[str, str]:
        if not self.circuit_open:
            try:
                start = time.perf_counter()
                resp = self.primary_model_fn(prompt)
                if time.perf_counter() - start > self.timeout_seconds:
                    raise TimeoutError("Primary LLM exceeded latency budget.")
                self.consecutive_failures = 0
                return resp, "primary_model"
            except Exception:
                self.consecutive_failures += 1
                if self.consecutive_failures >= self.failure_threshold:
                    self.circuit_open = True

        if self.fallback_model_fn:
            resp = self.fallback_model_fn(prompt)
            return resp, "fallback_model"
        raise RuntimeError("Primary model failed and no fallback available.")

    def parse_and_repair_json(self, raw_text: str, max_retries: int = 1) -> AnswerPayload:
        cleaned = raw_text.strip()
        if cleaned.startswith("```json"):
            cleaned = cleaned[7:]
        if cleaned.startswith("```"):
            cleaned = cleaned[3:]
        if cleaned.endswith("```"):
            cleaned = cleaned[:-3]
        cleaned = cleaned.strip()

        try:
            data = json.loads(cleaned)
            return AnswerPayload(**data)
        except Exception as err:
            if max_retries > 0:
                repair_prompt = f"Fix this invalid JSON to match the AnswerPayload schema. Errors: {err}\nInvalid JSON:\n{cleaned}"
                fixed_raw, _ = self.execute_inference_with_fallback(repair_prompt)
                return self.parse_and_repair_json(fixed_raw, max_retries=max_retries - 1)
            raise ValueError(f"Failed to parse structured JSON output: {err}")

if __name__ == "__main__":
    def mock_primary(p: str) -> str:
        return '{"summary": "SLA is 99.9%.", "detailed_points": ["Point 1"], "citations": ["doc1"], "confidence_score": 0.95}'
    engine = CoreAIEngine(primary_model_fn=mock_primary)
    prompt = engine.synthesize_prompt("What is SLA?", [{"id": "doc1", "text": "SLA is 99.9%."}])
    raw, prov = engine.execute_inference_with_fallback(prompt)
    print("Parsed:", engine.parse_and_repair_json(raw).summary)
metadata.yml (394 bytes)
lesson_id: D354
day: 354
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/core_ai.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 55
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (30 bytes)
pydantic>=2.0.0
pytest>=7.0.0
starter/core_ai.py (1989 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

import json
import time
from typing import Dict, Any, List, Tuple

class AnswerPayload:

    def __init__(self, summary: str, detailed_points: List[str], citations: List[str], confidence_score: float):
        raise NotImplementedError('TASK 1: implement __init__.')

    def model_dump(self) -> Dict[str, Any]:
        raise NotImplementedError('TASK 2: implement model_dump.')

class CoreAIEngine:

    def __init__(self, primary_model_fn, fallback_model_fn=None, failure_threshold: int=3, timeout_seconds: float=2.5):
        self.primary_model_fn = primary_model_fn
        self.fallback_model_fn = fallback_model_fn
        self.failure_threshold = failure_threshold
        self.timeout_seconds = timeout_seconds
        self.consecutive_failures = 0
        self.circuit_open = False

    def synthesize_prompt(self, user_query: str, context_chunks: List[Dict[str, Any]]) -> str:
        raise NotImplementedError('TASK 3: implement synthesize_prompt.')

    def execute_inference_with_fallback(self, prompt: str) -> Tuple[str, str]:
        raise NotImplementedError('TASK 4: implement execute_inference_with_fallback.')

    def parse_and_repair_json(self, raw_text: str, max_retries: int=1) -> AnswerPayload:
        raise NotImplementedError('TASK 5: implement parse_and_repair_json.')
if __name__ == '__main__':

    def mock_primary(p: str) -> str:
        raise NotImplementedError('TASK 6: implement mock_primary.')
    engine = CoreAIEngine(primary_model_fn=mock_primary)
    prompt = engine.synthesize_prompt('What is SLA?', [{'id': 'doc1', 'text': 'SLA is 99.9%.'}])
    raw, prov = engine.execute_inference_with_fallback(prompt)
    print('Parsed:', engine.parse_and_repair_json(raw).summary)
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_core_ai.py (2900 bytes)
import pytest
import time
from examples.core_ai import CoreAIEngine, AnswerPayload

def test_prompt_synthesis_formatting():
    engine = CoreAIEngine(primary_model_fn=lambda p: "")
    prompt = engine.synthesize_prompt("What is X?", [{"id": "doc1", "text": "Content of doc1"}])
    assert "<system_instruction>" in prompt
    assert "<doc id='doc1'>Content of doc1</doc>" in prompt
    assert "<user_query>" in prompt
    assert "What is X?" in prompt

def test_valid_json_schema_parsing():
    def mock_model(p: str) -> str:
        return '```json\n{"summary": "Result summary", "detailed_points": ["A", "B"], "citations": ["doc1"], "confidence_score": 0.92}\n```'
    
    engine = CoreAIEngine(primary_model_fn=mock_model)
    raw, provider = engine.execute_inference_with_fallback("query")
    parsed = engine.parse_and_repair_json(raw)
    assert isinstance(parsed, AnswerPayload)
    assert parsed.summary == "Result summary"
    assert parsed.confidence_score == 0.92
    assert len(parsed.detailed_points) == 2

def test_circuit_breaker_failover():
    def failing_primary(p: str) -> str:
        raise ConnectionError("Primary down")
        
    def healthy_fallback(p: str) -> str:
        return '{"summary": "Fallback answer", "detailed_points": [], "citations": [], "confidence_score": 0.80}'

    engine = CoreAIEngine(primary_model_fn=failing_primary, fallback_model_fn=healthy_fallback, failure_threshold=2)
    
    raw1, prov1 = engine.execute_inference_with_fallback("q")
    assert prov1 == "fallback_model"
    assert engine.circuit_open is False

    raw2, prov2 = engine.execute_inference_with_fallback("q")
    assert prov2 == "fallback_model"
    assert engine.circuit_open is True

def test_primary_timeout_detection():
    def slow_primary(p: str) -> str:
        time.sleep(0.05)
        return "{}"

    def fast_fallback(p: str) -> str:
        return '{"summary": "Fast fallback", "detailed_points": [], "citations": [], "confidence_score": 0.85}'

    engine = CoreAIEngine(primary_model_fn=slow_primary, fallback_model_fn=fast_fallback, timeout_seconds=0.01)
    raw, prov = engine.execute_inference_with_fallback("q")
    assert prov == "fallback_model"

def test_self_healing_repair_loop():
    call_count = 0
    def malformed_then_fixed(p: str) -> str:
        nonlocal call_count
        call_count += 1
        if call_count == 1:
            return '{"summary": "Incomplete JSON", "detailed_points": "not_a_list", "citations": [], "confidence_score": 0.5}'
        return '{"summary": "Fixed JSON", "detailed_points": ["Item 1"], "citations": ["doc1"], "confidence_score": 0.9}'

    engine = CoreAIEngine(primary_model_fn=malformed_then_fixed)
    raw, _ = engine.execute_inference_with_fallback("q")
    parsed = engine.parse_and_repair_json(raw, max_retries=1)
    assert parsed.summary == "Fixed JSON"
    assert isinstance(parsed.detailed_points, list)

Troubleshooting

Troubleshooting - Day 354\nEnsure markdown code block formatting is stripped before JSON validation.\n

Security notes

Security - Day 354\nCore AI engine isolates context in XML tags and validates structured JSON schemas.\n