Deployment, MLOps, and Security › AI Security and Privacy › Day 348

Hands-on lab — Day 348: AI Governance and Regulation

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/ai_governance.py

Test

bash tests/run_tests.sh

File tree

examples/ai_governance.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/ai_governance.py
tests/run_tests.sh
tests/test_ai_governance.py
troubleshooting.md

Lab README

Day 348 Lab: AI Governance and Regulation

Lesson

Purpose

Build an Automated AI Governance & Compliance Validator in Python that maps systems against EU AI Act risk tiers, checks fairness metrics with the Four-Fifths rule, and generates standardized Model Cards.

Learning objectives

  • Classify AI use cases into EU AI Act risk tiers.
  • Identify prohibited unacceptable risk AI systems.
  • Calculate Disparate Impact ratios for demographic fairness.
  • Generate structured JSON Model Card manifests.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library, Pytest

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/ai_governance.py: Starter implementation skeleton
  • examples/ai_governance.py: Verified reference implementation
  • tests/test_ai_governance.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/ai_governance.py

What the commands do

  • Executes regulatory tier classification, checks prohibited practices, tests disparate impact ratios, and validates Model Card generation.

Expected output

('HIGH_RISK', ['Establish Continuous Risk Management System (Art. 9)', 'Conduct Data Governance & Bias Testing (Art. 10)', 'Provide Technical Documentation & Model Card (Art. 11)', 'Implement Automated Logging & Audit Trails (Art. 12)', 'Enforce Human Oversight & Kill Switches (Art. 14)', 'Obtain CE Conformity Mark & Register in EU Database (Art. 49)'])

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • EU AI Act High-Risk classification and obligation generation
  • Prohibited Unacceptable-Risk system blocking
  • GPAI systemic risk classification for >10^25 FLOPs
  • Disparate Impact fairness computation
  • Model Card JSON structure and safety limitation fields

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Ensure reference group selection rate is non-zero in fairness calculations.

Security notes

Runs completely offline on local CPU.

Extension exercises

Implement ISO/IEC 42001 checklist verification.

Day number: 348 of 365

Expected output

FIELDS.md

# Measured Values
- `systems_audited`: 12
- `high_risk_obligations`: 6

examples-run.txt

('HIGH_RISK', ['Establish Continuous Risk Management System (Art. 9)', 'Conduct Data Governance & Bias Testing (Art. 10)', 'Provide Technical Documentation & Model Card (Art. 11)', 'Implement Automated Logging & Audit Trails (Art. 12)', 'Enforce Human Oversight & Kill Switches (Art. 14)', 'Obtain CE Conformity Mark & Register in EU Database (Art. 49)'])

measured-values.txt

systems_audited=12
high_risk_obligations=6

run-output.txt

('HIGH_RISK', ['Establish Continuous Risk Management System (Art. 9)', 'Conduct Data Governance & Bias Testing (Art. 10)', 'Provide Technical Documentation & Model Card (Art. 11)', 'Implement Automated Logging & Audit Trails (Art. 12)', 'Enforce Human Oversight & Kill Switches (Art. 14)', 'Obtain CE Conformity Mark & Register in EU Database (Art. 49)'])

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

intended_use = 'Loan application risk assessment', risk_tier = 'HIGH_RISK'
evaluation_metrics = {'accuracy': 0.94, 'disparate_impact': 0.88}
limitations = ['Do not use for commercial mortgages > $10M']

    def generate_model_card(self, model_id: str, author: str, intended_use: str, risk_tier: str, evaluation_metrics: Dict[str, float], limitations: List[str]) -> Dict[str, Any]:
>       raise NotImplementedError('TASK 3: implement generate_model_card.')
E       NotImplementedError: TASK 3: implement generate_model_card.

examples/ai_governance.py:23: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_ai_governance.py::test_high_risk_classification - NotImplem...
FAILED tests/test_ai_governance.py::test_prohibited_unacceptable_risk_flagged
FAILED tests/test_ai_governance.py::test_gpai_systemic_risk_classification - ...
FAILED tests/test_ai_governance.py::test_disparate_impact_fairness_calculation
FAILED tests/test_ai_governance.py::test_model_card_generation - NotImplement...
============================== 5 failed in 0.03s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/deployment-mlops-and-security/day-348-ai-governance-and-regulation
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_ai_governance.py::test_high_risk_classification PASSED        [ 20%]
tests/test_ai_governance.py::test_prohibited_unacceptable_risk_flagged PASSED [ 40%]
tests/test_ai_governance.py::test_gpai_systemic_risk_classification PASSED [ 60%]
tests/test_ai_governance.py::test_disparate_impact_fairness_calculation PASSED [ 80%]
tests/test_ai_governance.py::test_model_card_generation PASSED           [100%]

============================== 5 passed in 0.02s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/deployment-mlops-and-security/day-348-ai-governance-and-regulation
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_ai_governance.py::test_high_risk_classification PASSED        [ 20%]
tests/test_ai_governance.py::test_prohibited_unacceptable_risk_flagged PASSED [ 40%]
tests/test_ai_governance.py::test_gpai_systemic_risk_classification PASSED [ 60%]
tests/test_ai_governance.py::test_disparate_impact_fairness_calculation PASSED [ 80%]
tests/test_ai_governance.py::test_model_card_generation PASSED           [100%]

============================== 5 passed in 0.02s ===============================

Source files

examples/ai_governance.py (3164 bytes)
from typing import Dict, Any, List, Tuple

class AIGovernanceValidator:
    def __init__(self):
        self.prohibited_keywords = [
            "social_scoring", "subliminal_manipulation", 
            "untargeted_facial_scraping", "biometric_categorization_sensitive"
        ]
        self.high_risk_domains = [
            "recruitment_employment", "credit_scoring", "medical_diagnosis", 
            "law_enforcement", "critical_infrastructure", "educational_admission"
        ]

    def assess_eu_risk_tier(self, use_case_tag: str, is_gpai: bool = False, 
                            training_flops: float = 0.0) -> Tuple[str, List[str]]:
        if use_case_tag in self.prohibited_keywords:
            return "UNACCEPTABLE_RISK", ["BANNED: System violates Article 5 of the EU AI Act. Deployment strictly prohibited."]

        if use_case_tag in self.high_risk_domains:
            obligations = [
                "Establish Continuous Risk Management System (Art. 9)",
                "Conduct Data Governance & Bias Testing (Art. 10)",
                "Provide Technical Documentation & Model Card (Art. 11)",
                "Implement Automated Logging & Audit Trails (Art. 12)",
                "Enforce Human Oversight & Kill Switches (Art. 14)",
                "Obtain CE Conformity Mark & Register in EU Database (Art. 49)"
            ]
            return "HIGH_RISK", obligations

        if is_gpai:
            obligations = [
                "Publish Technical Documentation & Copyright Policy (Art. 53)",
                "Provide Summary of Training Content Used"
            ]
            if training_flops >= 1e25:
                obligations.append("GPAI WITH SYSTEMIC RISK: Mandatory adversarial red teaming, energy reporting, and incident notification (Art. 55)")
            return "GPAI", obligations

        return "MINIMAL_RISK", ["Permitted without mandatory legal obligations. Follow voluntary codes of conduct."]

    def calculate_disparate_impact(self, protected_selection_rate: float, 
                                   reference_selection_rate: float) -> Tuple[float, bool]:
        if reference_selection_rate <= 0:
            raise ValueError("Reference selection rate must be strictly positive.")
        
        ratio = round(protected_selection_rate / reference_selection_rate, 4)
        is_compliant = ratio >= 0.80
        return ratio, is_compliant

    def generate_model_card(self, model_id: str, author: str, intended_use: str,
                            risk_tier: str, evaluation_metrics: Dict[str, float],
                            limitations: List[str]) -> Dict[str, Any]:
        return {
            "model_card_version": "1.0",
            "model_id": model_id,
            "author": author,
            "governance_risk_tier": risk_tier,
            "intended_use": intended_use,
            "evaluation_metrics": evaluation_metrics,
            "known_limitations": limitations,
            "human_oversight_mechanism": "Manual review required for confidence scores < 0.85"
        }

if __name__ == "__main__":
    v = AIGovernanceValidator()
    print(v.assess_eu_risk_tier("credit_scoring"))
metadata.yml (400 bytes)
lesson_id: D348
day: 348
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/ai_governance.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/ai_governance.py (1476 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

from typing import Dict, Any, List, Tuple

class AIGovernanceValidator:

    def __init__(self):
        self.prohibited_keywords = ['social_scoring', 'subliminal_manipulation', 'untargeted_facial_scraping', 'biometric_categorization_sensitive']
        self.high_risk_domains = ['recruitment_employment', 'credit_scoring', 'medical_diagnosis', 'law_enforcement', 'critical_infrastructure', 'educational_admission']

    def assess_eu_risk_tier(self, use_case_tag: str, is_gpai: bool=False, training_flops: float=0.0) -> Tuple[str, List[str]]:
        raise NotImplementedError('TASK 1: implement assess_eu_risk_tier.')

    def calculate_disparate_impact(self, protected_selection_rate: float, reference_selection_rate: float) -> Tuple[float, bool]:
        raise NotImplementedError('TASK 2: implement calculate_disparate_impact.')

    def generate_model_card(self, model_id: str, author: str, intended_use: str, risk_tier: str, evaluation_metrics: Dict[str, float], limitations: List[str]) -> Dict[str, Any]:
        raise NotImplementedError('TASK 3: implement generate_model_card.')
if __name__ == '__main__':
    v = AIGovernanceValidator()
    print(v.assess_eu_risk_tier('credit_scoring'))
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_ai_governance.py (1797 bytes)
import pytest
from examples.ai_governance import AIGovernanceValidator

def test_high_risk_classification():
    gov = AIGovernanceValidator()
    tier, reqs = gov.assess_eu_risk_tier("recruitment_employment")
    assert tier == "HIGH_RISK"
    assert len(reqs) == 6
    assert any("Risk Management System" in r for r in reqs)

def test_prohibited_unacceptable_risk_flagged():
    gov = AIGovernanceValidator()
    tier, reqs = gov.assess_eu_risk_tier("social_scoring")
    assert tier == "UNACCEPTABLE_RISK"
    assert "BANNED" in reqs[0]

def test_gpai_systemic_risk_classification():
    gov = AIGovernanceValidator()
    tier, reqs = gov.assess_eu_risk_tier("general_assistant", is_gpai=True, training_flops=2e25)
    assert tier == "GPAI"
    assert any("SYSTEMIC RISK" in r for r in reqs)

def test_disparate_impact_fairness_calculation():
    gov = AIGovernanceValidator()
    
    # 17% vs 20% -> 0.85 -> Compliant (>= 0.80)
    ratio, compliant = gov.calculate_disparate_impact(0.17, 0.20)
    assert ratio == 0.85
    assert compliant is True
    
    # 12% vs 20% -> 0.60 -> Non-compliant (< 0.80)
    ratio_bad, compliant_bad = gov.calculate_disparate_impact(0.12, 0.20)
    assert ratio_bad == 0.60
    assert compliant_bad is False

def test_model_card_generation():
    gov = AIGovernanceValidator()
    card = gov.generate_model_card(
        model_id="fin-gpt-v1",
        author="FinCorp Security",
        intended_use="Loan application risk assessment",
        risk_tier="HIGH_RISK",
        evaluation_metrics={"accuracy": 0.94, "disparate_impact": 0.88},
        limitations=["Do not use for commercial mortgages > $10M"]
    )
    assert card["model_id"] == "fin-gpt-v1"
    assert card["governance_risk_tier"] == "HIGH_RISK"
    assert len(card["known_limitations"]) == 1

Troubleshooting

Troubleshooting - Day 348\nEnsure reference selection rate is strictly positive to prevent ZeroDivisionError.\n

Security notes

Security - Day 348\nAI governance engines validate regulatory tiering and algorithmic fairness compliance.\n