Deployment, MLOps, and Security › AI Security and Privacy › Day 344

Hands-on lab — Day 344: Threat Modeling AI Systems

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/threat_modeling.py

Test

bash tests/run_tests.sh

File tree

examples/threat_modeling.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/threat_modeling.py
tests/run_tests.sh
tests/test_threat_modeling.py
troubleshooting.md

Lab README

Day 344 Lab: Threat Modeling AI Systems

Lesson

Purpose

Build an AI Threat Modeling & DREAD Risk Scorer in Python calculating composite risk priorities, categorizing vulnerabilities under STRIDE and OWASP LLM, and generating prioritized remediation plans.

Learning objectives

  • Register threat vectors with STRIDE categories and OWASP Top 10 mappings.
  • Validate DREAD criteria scores (1–10).
  • Compute composite risk scores and assign severity ratings.
  • Generate prioritized remediation plans.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library, Pytest

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/threat_modeling.py: Starter implementation skeleton
  • examples/threat_modeling.py: Verified reference implementation
  • tests/test_threat_modeling.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/threat_modeling.py

What the commands do

  • Evaluates DREAD criteria, verifies risk score calculations, and tests remediation plan sorting.

Expected output

{'threat_id': 'T1', 'title': 'Prompt Injection', 'stride_category': 'Elevation', 'owasp_id': 'LLM01', 'dread_scores': {'damage': 9, 'reproducibility': 8, 'exploitability': 8, 'affected_users': 9, 'discoverability': 8}, 'composite_score': 8.4, 'severity': 'CRITICAL', 'description': 'Test'}

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Correct composite DREAD score calculation
  • Severity tier classification (CRITICAL, HIGH, MEDIUM, LOW)
  • DREAD input boundary validation (1–10 integer range)
  • Prioritized remediation plan sorting descending by composite risk
  • Empty threat model report handling

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Ensure all DREAD criteria integers are between 1 and 10 inclusive.

Security notes

Runs completely offline on local CPU.

Extension exercises

Implement a return-on-security-investment (ROSI) score calculator.

Day number: 344 of 365

Expected output

FIELDS.md

# Measured Values
- `threats_modeled`: 8
- `critical_threats`: 2

examples-run.txt

{'threat_id': 'T1', 'title': 'Prompt Injection', 'stride_category': 'Elevation', 'owasp_id': 'LLM01', 'dread_scores': {'damage': 9, 'reproducibility': 8, 'exploitability': 8, 'affected_users': 9, 'discoverability': 8}, 'composite_score': 8.4, 'severity': 'CRITICAL', 'description': 'Test'}

measured-values.txt

threats_modeled=8
critical_threats=2

run-output.txt

{'threat_id': 'T1', 'title': 'Prompt Injection', 'stride_category': 'Elevation', 'owasp_id': 'LLM01', 'dread_scores': {'damage': 9, 'reproducibility': 8, 'exploitability': 8, 'affected_users': 9, 'discoverability': 8}, 'composite_score': 8.4, 'severity': 'CRITICAL', 'description': 'Test'}

starter-run.txt

An untouched starter fails the suite. Captured from a real run:

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <examples.threat_modeling.AIThreatModelScorer object at 0x107406780>

    def get_prioritized_remediation_plan(self) -> List[Dict[str, Any]]:
>       raise NotImplementedError('TASK 2: implement get_prioritized_remediation_plan.')
E       NotImplementedError: TASK 2: implement get_prioritized_remediation_plan.

examples/threat_modeling.py:19: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_threat_modeling.py::test_dread_score_calculation_and_severity
FAILED tests/test_threat_modeling.py::test_severity_tier_classification - Not...
FAILED tests/test_threat_modeling.py::test_invalid_dread_criteria_raises_error
FAILED tests/test_threat_modeling.py::test_prioritized_remediation_sorting - ...
FAILED tests/test_threat_modeling.py::test_empty_threat_model - NotImplemente...
============================== 5 failed in 0.03s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/deployment-mlops-and-security/day-344-threat-modeling-ai-systems
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_threat_modeling.py::test_dread_score_calculation_and_severity PASSED [ 20%]
tests/test_threat_modeling.py::test_severity_tier_classification PASSED  [ 40%]
tests/test_threat_modeling.py::test_invalid_dread_criteria_raises_error PASSED [ 60%]
tests/test_threat_modeling.py::test_prioritized_remediation_sorting PASSED [ 80%]
tests/test_threat_modeling.py::test_empty_threat_model PASSED            [100%]

============================== 5 passed in 0.01s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/deployment-mlops-and-security/day-344-threat-modeling-ai-systems
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_threat_modeling.py::test_dread_score_calculation_and_severity PASSED [ 20%]
tests/test_threat_modeling.py::test_severity_tier_classification PASSED  [ 40%]
tests/test_threat_modeling.py::test_invalid_dread_criteria_raises_error PASSED [ 60%]
tests/test_threat_modeling.py::test_prioritized_remediation_sorting PASSED [ 80%]
tests/test_threat_modeling.py::test_empty_threat_model PASSED            [100%]

============================== 5 passed in 0.01s ===============================

Source files

examples/threat_modeling.py (1993 bytes)
from typing import Dict, Any, List, Optional

class AIThreatModelScorer:
    def __init__(self):
        self.identified_threats: List[Dict[str, Any]] = []

    def add_threat(self, threat_id: str, title: str, category_stride: str, 
                   owasp_id: str, damage: int, reproducibility: int, 
                   exploitability: int, affected_users: int, discoverability: int,
                   description: str) -> Dict[str, Any]:
        scores = [damage, reproducibility, exploitability, affected_users, discoverability]
        for s in scores:
            if not isinstance(s, int) or not (1 <= s <= 10):
                raise ValueError("All DREAD criteria must be integers between 1 and 10.")

        composite_score = round(sum(scores) / 5.0, 2)
        
        if composite_score >= 7.5:
            severity = "CRITICAL"
        elif composite_score >= 5.0:
            severity = "HIGH"
        elif composite_score >= 3.0:
            severity = "MEDIUM"
        else:
            severity = "LOW"

        record = {
            "threat_id": threat_id,
            "title": title,
            "stride_category": category_stride,
            "owasp_id": owasp_id,
            "dread_scores": {
                "damage": damage,
                "reproducibility": reproducibility,
                "exploitability": exploitability,
                "affected_users": affected_users,
                "discoverability": discoverability
            },
            "composite_score": composite_score,
            "severity": severity,
            "description": description
        }
        self.identified_threats.append(record)
        return record

    def get_prioritized_remediation_plan(self) -> List[Dict[str, Any]]:
        return sorted(self.identified_threats, key=lambda x: x["composite_score"], reverse=True)

if __name__ == "__main__":
    s = AIThreatModelScorer()
    print(s.add_threat("T1", "Prompt Injection", "Elevation", "LLM01", 9, 8, 8, 9, 8, "Test"))
metadata.yml (402 bytes)
lesson_id: D344
day: 344
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/threat_modeling.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/threat_modeling.py (1042 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

from typing import Dict, Any, List, Optional

class AIThreatModelScorer:

    def __init__(self):
        self.identified_threats: List[Dict[str, Any]] = []

    def add_threat(self, threat_id: str, title: str, category_stride: str, owasp_id: str, damage: int, reproducibility: int, exploitability: int, affected_users: int, discoverability: int, description: str) -> Dict[str, Any]:
        raise NotImplementedError('TASK 1: implement add_threat.')

    def get_prioritized_remediation_plan(self) -> List[Dict[str, Any]]:
        raise NotImplementedError('TASK 2: implement get_prioritized_remediation_plan.')
if __name__ == '__main__':
    s = AIThreatModelScorer()
    print(s.add_threat('T1', 'Prompt Injection', 'Elevation', 'LLM01', 9, 8, 8, 9, 8, 'Test'))
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_threat_modeling.py (2051 bytes)
import pytest
from examples.threat_modeling import AIThreatModelScorer

def test_dread_score_calculation_and_severity():
    scorer = AIThreatModelScorer()
    # 9, 8, 8, 9, 8 -> sum=42 / 5 = 8.4 -> CRITICAL
    t = scorer.add_threat("T1", "Indirect Injection", "Tampering", "LLM01", 9, 8, 8, 9, 8, "Exploit")
    
    assert t["composite_score"] == 8.4
    assert t["severity"] == "CRITICAL"
    assert t["owasp_id"] == "LLM01"

def test_severity_tier_classification():
    scorer = AIThreatModelScorer()
    # High: 6, 6, 6, 6, 6 -> 6.0
    t_high = scorer.add_threat("T2", "Prompt Leak", "Info Disc", "LLM06", 6, 6, 6, 6, 6, "Desc")
    assert t_high["severity"] == "HIGH"
    
    # Medium: 4, 4, 4, 4, 4 -> 4.0
    t_med = scorer.add_threat("T3", "DoS", "DoS", "LLM04", 4, 4, 4, 4, 4, "Desc")
    assert t_med["severity"] == "MEDIUM"
    
    # Low: 2, 2, 2, 2, 2 -> 2.0
    t_low = scorer.add_threat("T4", "Minor", "Info", "LLM09", 2, 2, 2, 2, 2, "Desc")
    assert t_low["severity"] == "LOW"

def test_invalid_dread_criteria_raises_error():
    scorer = AIThreatModelScorer()
    with pytest.raises(ValueError):
        scorer.add_threat("T_ERR", "Bad Score", "Spoofing", "LLM01", 11, 5, 5, 5, 5, "Out of bounds")
        
    with pytest.raises(ValueError):
        scorer.add_threat("T_ERR2", "Bad Score 0", "Spoofing", "LLM01", 0, 5, 5, 5, 5, "Zero score")

def test_prioritized_remediation_sorting():
    scorer = AIThreatModelScorer()
    scorer.add_threat("T_LOW", "Low Risk", "DoS", "LLM04", 2, 2, 2, 2, 2, "Low")
    scorer.add_threat("T_CRIT", "Crit Risk", "Elevation", "LLM01", 10, 10, 9, 10, 9, "Crit")
    scorer.add_threat("T_HIGH", "High Risk", "Info", "LLM06", 6, 6, 6, 6, 6, "High")
    
    plan = scorer.get_prioritized_remediation_plan()
    assert len(plan) == 3
    assert plan[0]["threat_id"] == "T_CRIT"
    assert plan[1]["threat_id"] == "T_HIGH"
    assert plan[2]["threat_id"] == "T_LOW"

def test_empty_threat_model():
    scorer = AIThreatModelScorer()
    assert len(scorer.get_prioritized_remediation_plan()) == 0

Troubleshooting

Troubleshooting - Day 344\nEnsure DREAD scores are positive integers between 1 and 10.\n

Security notes

Security - Day 344\nThreat modeling enforces STRIDE trust boundaries and quantitative DREAD severity bounds.\n