Deployment, MLOps, and Security › AI Security and Privacy › Day 347

Hands-on lab — Day 347: Model and Supply Chain Security

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/model_security.py

Test

bash tests/run_tests.sh

File tree

examples/model_security.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/model_security.py
tests/run_tests.sh
tests/test_model_security.py
troubleshooting.md

Lab README

Day 347 Lab: Model and Supply Chain Security

Lesson

Purpose

Build a Production Model Supply Chain Security Scanner in Python that audits model directories, blocks dangerous pickle formats, calculates SHA256 checksums, and generates an AI Software Bill of Materials (AIBOM).

Learning objectives

  • Identify and flag insecure serialized model files (.pt, .pkl, .bin).
  • Calculate SHA256 cryptographic hashes for supply chain validation.
  • Validate SafeTensors compliance across model weight bundles.
  • Generate structured CycloneDX-AI bill of materials manifests.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library, Pytest

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/model_security.py: Starter implementation skeleton
  • examples/model_security.py: Verified reference implementation
  • tests/test_model_security.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/model_security.py

What the commands do

  • Executes directory scans, checks format validation rules, verifies SHA256 hashes, and tests AIBOM JSON generation.

Expected output

Scanner ready.

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Insecure pickle format detection and rejection
  • SafeTensors compliance verification
  • SHA256 digest computation accuracy
  • AIBOM CycloneDX manifest generation
  • Non-existent directory error handling

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Ensure all files are opened in binary mode ('rb') for hashing.

Security notes

Runs completely offline on local CPU.

Extension exercises

Implement an automated converter from .pt to .safetensors.

Day number: 347 of 365

Expected output

FIELDS.md

# Measured Values
- `safetensors_scanned`: 10
- `insecure_pickles_blocked`: 2

examples-run.txt

Scanner ready.

measured-values.txt

safetensors_scanned=10
insecure_pickles_blocked=2

run-output.txt

Scanner ready.

starter-run.txt

An untouched starter fails the suite. Captured from a real run:


self = <examples.model_security.ModelSupplyChainScanner object at 0x106fbe780>
dir_path = '/path/to/nonexistent/dir/12345'

    def scan_model_directory(self, dir_path: str) -> Tuple[bool, List[str], Dict[str, Any]]:
>       raise NotImplementedError('TASK 2: implement scan_model_directory.')
E       NotImplementedError: TASK 2: implement scan_model_directory.

examples/model_security.py:22: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_model_security.py::test_insecure_pickle_format_detected - N...
FAILED tests/test_model_security.py::test_safetensors_compliance - NotImpleme...
FAILED tests/test_model_security.py::test_sha256_hash_calculation - NotImplem...
FAILED tests/test_model_security.py::test_aibom_manifest_structure - NotImple...
FAILED tests/test_model_security.py::test_nonexistent_directory - NotImplemen...
============================== 5 failed in 0.04s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/deployment-mlops-and-security/day-347-model-and-supply-chain-security
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_model_security.py::test_insecure_pickle_format_detected PASSED [ 20%]
tests/test_model_security.py::test_safetensors_compliance PASSED         [ 40%]
tests/test_model_security.py::test_sha256_hash_calculation PASSED        [ 60%]
tests/test_model_security.py::test_aibom_manifest_structure PASSED       [ 80%]
tests/test_model_security.py::test_nonexistent_directory PASSED          [100%]

============================== 5 passed in 0.02s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/deployment-mlops-and-security/day-347-model-and-supply-chain-security
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_model_security.py::test_insecure_pickle_format_detected PASSED [ 20%]
tests/test_model_security.py::test_safetensors_compliance PASSED         [ 40%]
tests/test_model_security.py::test_sha256_hash_calculation PASSED        [ 60%]
tests/test_model_security.py::test_aibom_manifest_structure PASSED       [ 80%]
tests/test_model_security.py::test_nonexistent_directory PASSED          [100%]

============================== 5 passed in 0.02s ===============================

Source files

examples/model_security.py (2303 bytes)
import os
import json
import hashlib
from typing import Dict, Any, List, Tuple

class ModelSupplyChainScanner:
    def __init__(self, allowed_extensions: Tuple[str, ...] = (".safetensors", ".json", ".txt")):
        self.allowed_extensions = allowed_extensions

    def calculate_sha256(self, file_path: str) -> str:
        sha256 = hashlib.sha256()
        with open(file_path, "rb") as f:
            while chunk := f.read(65536):
                sha256.update(chunk)
        return sha256.hexdigest()

    def scan_model_directory(self, dir_path: str) -> Tuple[bool, List[str], Dict[str, Any]]:
        findings: List[str] = []
        is_compliant = True
        artifacts: List[Dict[str, Any]] = []

        if not os.path.exists(dir_path):
            return False, [f"Directory not found: {dir_path}"], {}

        for root, _, files in os.walk(dir_path):
            for file in sorted(files):
                full_path = os.path.join(root, file)
                ext = os.path.splitext(file)[1].lower()

                if ext in [".pt", ".bin", ".pkl", ".joblib", ".pickle"]:
                    is_compliant = False
                    findings.append(f"CRITICAL: Insecure serialized format detected: {file} ({ext}). Must convert to .safetensors.")

                if ext not in self.allowed_extensions:
                    findings.append(f"WARNING: Unrecognized file extension in model bundle: {file}")

                file_hash = self.calculate_sha256(full_path)
                file_size = os.path.getsize(full_path)
                artifacts.append({
                    "filename": file,
                    "relative_path": os.path.relpath(full_path, dir_path),
                    "size_bytes": file_size,
                    "sha256": file_hash,
                    "format": "safetensors" if ext == ".safetensors" else "metadata"
                })

        aibom = {
            "bom_format": "CycloneDX-AI",
            "spec_version": "1.5",
            "model_name": os.path.basename(dir_path.rstrip("/\\")),
            "compliant": is_compliant,
            "total_artifacts": len(artifacts),
            "artifacts": artifacts
        }

        return is_compliant, findings, aibom

if __name__ == "__main__":
    s = ModelSupplyChainScanner()
    print("Scanner ready.")
metadata.yml (401 bytes)
lesson_id: D347
day: 347
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/model_security.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/model_security.py (926 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

import os
import json
import hashlib
from typing import Dict, Any, List, Tuple

class ModelSupplyChainScanner:

    def __init__(self, allowed_extensions: Tuple[str, ...]=('.safetensors', '.json', '.txt')):
        self.allowed_extensions = allowed_extensions

    def calculate_sha256(self, file_path: str) -> str:
        raise NotImplementedError('TASK 1: implement calculate_sha256.')

    def scan_model_directory(self, dir_path: str) -> Tuple[bool, List[str], Dict[str, Any]]:
        raise NotImplementedError('TASK 2: implement scan_model_directory.')
if __name__ == '__main__':
    s = ModelSupplyChainScanner()
    print('Scanner ready.')
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_model_security.py (2562 bytes)
import os
import tempfile
import pytest
from examples.model_security import ModelSupplyChainScanner

def test_insecure_pickle_format_detected():
    scanner = ModelSupplyChainScanner()
    with tempfile.TemporaryDirectory() as tmpdir:
        # Create insecure .pt file
        with open(os.path.join(tmpdir, "weights.pt"), "wb") as f:
            f.write(b"fake pickle payload")
            
        compliant, findings, aibom = scanner.scan_model_directory(tmpdir)
        assert compliant is False
        assert any("CRITICAL: Insecure serialized format detected" in f for f in findings)
        assert aibom["compliant"] is False

def test_safetensors_compliance():
    scanner = ModelSupplyChainScanner()
    with tempfile.TemporaryDirectory() as tmpdir:
        # Create safe safetensors file
        with open(os.path.join(tmpdir, "model.safetensors"), "wb") as f:
            f.write(b"safetensors data buffer")
        with open(os.path.join(tmpdir, "config.json"), "w") as f:
            f.write('{"vocab_size": 32000}')
            
        compliant, findings, aibom = scanner.scan_model_directory(tmpdir)
        assert compliant is True
        assert len(findings) == 0
        assert aibom["compliant"] is True
        assert aibom["total_artifacts"] == 2

def test_sha256_hash_calculation():
    scanner = ModelSupplyChainScanner()
    with tempfile.TemporaryDirectory() as tmpdir:
        test_file = os.path.join(tmpdir, "test.txt")
        with open(test_file, "wb") as f:
            f.write(b"hello world")
            
        # SHA256 for 'hello world' is b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9
        h = scanner.calculate_sha256(test_file)
        assert h == "b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde9"

def test_aibom_manifest_structure():
    scanner = ModelSupplyChainScanner()
    with tempfile.TemporaryDirectory() as tmpdir:
        with open(os.path.join(tmpdir, "weights.safetensors"), "wb") as f:
            f.write(b"bytes")
            
        _, _, aibom = scanner.scan_model_directory(tmpdir)
        assert aibom["bom_format"] == "CycloneDX-AI"
        assert "artifacts" in aibom
        assert aibom["artifacts"][0]["filename"] == "weights.safetensors"
        assert aibom["artifacts"][0]["format"] == "safetensors"

def test_nonexistent_directory():
    scanner = ModelSupplyChainScanner()
    compliant, findings, aibom = scanner.scan_model_directory("/path/to/nonexistent/dir/12345")
    assert compliant is False
    assert "Directory not found" in findings[0]

Troubleshooting

Troubleshooting - Day 347\nEnsure files are read in binary mode when calculating SHA256 hashes.\n

Security notes

Security - Day 347\nModel supply chain scanners enforce SafeTensors formats and SHA256 verification.\n