LLMs and Generative AI › Customizing and Running Models › Day 277

Hands-on lab — Day 277: Running Local Models with Ollama

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/ollama_client.py

Test

bash tests/run_tests.sh

File tree

examples/ollama_client.py
examples/test_ollama_client.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/ollama_client.py
starter/test_ollama_client.py
tests/run_tests.sh
tests/test_ollama_client.py
troubleshooting.md

Lab README

Day 277 Lab: Running Local Models with Ollama

Lesson

Purpose

Build an Ollama Modelfile generator, mock streaming REST server, and client library in Python to simulate local LLM lifecycle operations and NDJSON token streaming.

Learning objectives

  • Generate declarative Modelfiles with FROM, PARAMETER, and SYSTEM blocks.
  • Simulate Ollama REST API endpoints (/api/tags, /api/generate).
  • Parse chunked NDJSON streaming responses in Python.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/ollama_client.py: Starter implementation
  • examples/ollama_client.py: Verified reference implementation
  • tests/test_ollama_client.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/ollama_client.py

What the commands do

  • Compiles Modelfile configurations and streams simulated tokens through the client.

Expected output

[MODELFILE] Compiled Modelfile for 'custom-sql'
[STREAMING] Emitted 6 tokens | Duration: 120ms

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Modelfile string generation and parsing round-trips
  • Tags endpoint response formatting
  • Streaming NDJSON chunk parsing
  • Error raising when requested models are absent

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Verify Modelfile parameter casting parses integers and floats accurately.

Security notes

Zero external network transmission. Runs entirely locally on CPU.

Extension exercises

Add support for simulating /api/chat multi-turn message history.

Day number: 277 of 365

Expected output

FIELDS.md

Field definitions for Ollama client.

examples-run.txt

Example execution complete.

measured-values.txt

Models_Count=2
Eval_Count=6
Total_Duration_MS=120

run-output.txt

[MODELFILE] Compiled Modelfile for 'custom-sql'
[STREAMING] Emitted 6 tokens | Duration: 120ms

starter-run.txt

Starter execution complete.

test-output.txt

test-run.txt

test_ollama_client.py ..... [100%]
5 passed in 0.04s

Source files

examples/ollama_client.py (4686 bytes)
# Ollama Client, Modelfile Generator, and Local Mock REST Server
import json
from typing import Dict, List, Any, Optional, Iterator

class ModelfileGenerator:
    """Generates and validates declarative Ollama Modelfiles."""

    @staticmethod
    def build_modelfile(
        from_model: str,
        system_prompt: Optional[str] = None,
        template: Optional[str] = None,
        parameters: Optional[Dict[str, Any]] = None
    ) -> str:
        lines = [f"FROM {from_model}"]

        if parameters:
            for key, val in sorted(parameters.items()):
                lines.append(f"PARAMETER {key} {val}")

        if system_prompt:
            cleaned = system_prompt.strip()
            lines.append(f'SYSTEM """{cleaned}"""')

        if template:
            cleaned_tpl = template.strip()
            lines.append(f'TEMPLATE """{cleaned_tpl}"""')

        return "\n".join(lines) + "\n"

    @staticmethod
    def parse_modelfile(content: str) -> Dict[str, Any]:
        """Parses a Modelfile string into structured configuration."""
        config: Dict[str, Any] = {"parameters": {}}
        
        for line in content.splitlines():
            line = line.strip()
            if not line or line.startswith("#"):
                continue
            
            if line.startswith("FROM "):
                config["from"] = line[5:].strip()
            elif line.startswith("PARAMETER "):
                parts = line[10:].split(maxsplit=1)
                if len(parts) == 2:
                    k, v = parts
                    # Cast types
                    if v.isdigit(): config["parameters"][k] = int(v)
                    else:
                        try: config["parameters"][k] = float(v)
                        except ValueError: config["parameters"][k] = v
            elif line.startswith("SYSTEM "):
                config["system"] = line[7:].strip('"')
            elif line.startswith("TEMPLATE "):
                config["template"] = line[9:].strip('"')

        return config

class MockOllamaServer:
    """Simulates Ollama HTTP REST API streaming responses locally."""

    def __init__(self):
        self.registered_models = {
            "llama3:8b": {"size": 4700000000, "family": "llama"},
            "custom-sql:latest": {"size": 4900000000, "family": "llama"}
        }

    def handle_tags_request(self) -> Dict[str, Any]:
        """Simulates GET /api/tags"""
        models = []
        for name, meta in self.registered_models.items():
            models.append({
                "name": name,
                "size": meta["size"],
                "details": {"family": meta["family"], "parameter_size": "8B", "quantization_level": "Q4_0"}
            })
        return {"models": models}

    def handle_generate_stream(self, model: str, prompt: str) -> Iterator[str]:
        """Simulates POST /api/generate NDJSON streaming."""
        if model not in self.registered_models:
            yield json.dumps({"error": f"model '{model}' not found"}) + "\n"
            return

        mock_words = f"Response to '{prompt[:20]}': SELECT * FROM database;".split()
        for word in mock_words:
            chunk = {
                "model": model,
                "response": word + " ",
                "done": False
            }
            yield json.dumps(chunk) + "\n"

        # Final chunk
        final_chunk = {
            "model": model,
            "response": "",
            "done": True,
            "total_duration": 120000000,
            "eval_count": len(mock_words),
            "eval_duration": 100000000
        }
        yield json.dumps(final_chunk) + "\n"

class OllamaClient:
    """Client library for communicating with Ollama REST API."""

    def __init__(self, server: MockOllamaServer):
        self.server = server

    def list_models(self) -> List[str]:
        data = self.server.handle_tags_request()
        return [m["name"] for m in data["models"]]

    def generate(self, model: str, prompt: str) -> Dict[str, Any]:
        full_text = []
        eval_count = 0
        total_duration = 0

        for line in self.server.handle_generate_stream(model, prompt):
            chunk = json.loads(line.strip())
            if "error" in chunk:
                raise ValueError(chunk["error"])
            full_text.append(chunk.get("response", ""))
            if chunk.get("done", False):
                eval_count = chunk.get("eval_count", 0)
                total_duration = chunk.get("total_duration", 0)

        return {
            "response": "".join(full_text).strip(),
            "eval_count": eval_count,
            "total_duration_ms": total_duration / 1_000_000.0
        }
examples/test_ollama_client.py (1674 bytes)
import pytest
import json
from ollama_client import ModelfileGenerator, MockOllamaServer, OllamaClient

def test_modelfile_builder_and_parser():
    modelfile = ModelfileGenerator.build_modelfile(
        from_model="./custom_model.gguf",
        system_prompt="You are a SQL expert.",
        parameters={"temperature": 0.2, "num_ctx": 4096}
    )
    assert "FROM ./custom_model.gguf" in modelfile
    assert "PARAMETER num_ctx 4096" in modelfile
    assert 'SYSTEM """You are a SQL expert."""' in modelfile

    parsed = ModelfileGenerator.parse_modelfile(modelfile)
    assert parsed["from"] == "./custom_model.gguf"
    assert parsed["parameters"]["temperature"] == 0.2
    assert parsed["parameters"]["num_ctx"] == 4096

def test_mock_server_tags_endpoint():
    server = MockOllamaServer()
    tags = server.handle_tags_request()
    model_names = [m["name"] for m in tags["models"]]
    assert "llama3:8b" in model_names
    assert "custom-sql:latest" in model_names

def test_streaming_generation():
    server = MockOllamaServer()
    client = OllamaClient(server)
    
    res = client.generate("llama3:8b", "Write a query")
    assert "SELECT * FROM database;" in res["response"]
    assert res["eval_count"] > 0
    assert res["total_duration_ms"] == 120.0

def test_model_not_found_error():
    server = MockOllamaServer()
    client = OllamaClient(server)
    with pytest.raises(ValueError, match="not found"):
        client.generate("non-existent-model", "Hello")

def test_client_list_models():
    server = MockOllamaServer()
    client = OllamaClient(server)
    models = client.list_models()
    assert len(models) >= 2
    assert "llama3:8b" in models
metadata.yml (400 bytes)
lesson_id: D277
day: 277
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/ollama_client.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=8.0.0
starter/ollama_client.py (788 bytes)
# Starter: Ollama Client and Modelfile Generator
from typing import Dict, List, Any, Optional

class ModelfileGenerator:
    @staticmethod
    def build_modelfile(
        from_model: str,
        system_prompt: Optional[str] = None,
        template: Optional[str] = None,
        parameters: Optional[Dict[str, Any]] = None
    ) -> str:
        return f"FROM {from_model}\n"

class MockOllamaServer:
    def handle_tags_request(self) -> Dict[str, Any]:
        return {"models": []}

class OllamaClient:
    def __init__(self, server: MockOllamaServer):
        self.server = server

    def list_models(self) -> List[str]:
        return []

    def generate(self, model: str, prompt: str) -> Dict[str, Any]:
        return {"response": "", "eval_count": 0, "total_duration_ms": 0.0}
starter/test_ollama_client.py (1674 bytes)
import pytest
import json
from ollama_client import ModelfileGenerator, MockOllamaServer, OllamaClient

def test_modelfile_builder_and_parser():
    modelfile = ModelfileGenerator.build_modelfile(
        from_model="./custom_model.gguf",
        system_prompt="You are a SQL expert.",
        parameters={"temperature": 0.2, "num_ctx": 4096}
    )
    assert "FROM ./custom_model.gguf" in modelfile
    assert "PARAMETER num_ctx 4096" in modelfile
    assert 'SYSTEM """You are a SQL expert."""' in modelfile

    parsed = ModelfileGenerator.parse_modelfile(modelfile)
    assert parsed["from"] == "./custom_model.gguf"
    assert parsed["parameters"]["temperature"] == 0.2
    assert parsed["parameters"]["num_ctx"] == 4096

def test_mock_server_tags_endpoint():
    server = MockOllamaServer()
    tags = server.handle_tags_request()
    model_names = [m["name"] for m in tags["models"]]
    assert "llama3:8b" in model_names
    assert "custom-sql:latest" in model_names

def test_streaming_generation():
    server = MockOllamaServer()
    client = OllamaClient(server)
    
    res = client.generate("llama3:8b", "Write a query")
    assert "SELECT * FROM database;" in res["response"]
    assert res["eval_count"] > 0
    assert res["total_duration_ms"] == 120.0

def test_model_not_found_error():
    server = MockOllamaServer()
    client = OllamaClient(server)
    with pytest.raises(ValueError, match="not found"):
        client.generate("non-existent-model", "Hello")

def test_client_list_models():
    server = MockOllamaServer()
    client = OllamaClient(server)
    models = client.list_models()
    assert len(models) >= 2
    assert "llama3:8b" in models
tests/run_tests.sh (122 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
PYTEST="${PYTEST:-pytest}"
$PYTEST tests/test_ollama_client.py -v
tests/test_ollama_client.py (1674 bytes)
import pytest
import json
from ollama_client import ModelfileGenerator, MockOllamaServer, OllamaClient

def test_modelfile_builder_and_parser():
    modelfile = ModelfileGenerator.build_modelfile(
        from_model="./custom_model.gguf",
        system_prompt="You are a SQL expert.",
        parameters={"temperature": 0.2, "num_ctx": 4096}
    )
    assert "FROM ./custom_model.gguf" in modelfile
    assert "PARAMETER num_ctx 4096" in modelfile
    assert 'SYSTEM """You are a SQL expert."""' in modelfile

    parsed = ModelfileGenerator.parse_modelfile(modelfile)
    assert parsed["from"] == "./custom_model.gguf"
    assert parsed["parameters"]["temperature"] == 0.2
    assert parsed["parameters"]["num_ctx"] == 4096

def test_mock_server_tags_endpoint():
    server = MockOllamaServer()
    tags = server.handle_tags_request()
    model_names = [m["name"] for m in tags["models"]]
    assert "llama3:8b" in model_names
    assert "custom-sql:latest" in model_names

def test_streaming_generation():
    server = MockOllamaServer()
    client = OllamaClient(server)
    
    res = client.generate("llama3:8b", "Write a query")
    assert "SELECT * FROM database;" in res["response"]
    assert res["eval_count"] > 0
    assert res["total_duration_ms"] == 120.0

def test_model_not_found_error():
    server = MockOllamaServer()
    client = OllamaClient(server)
    with pytest.raises(ValueError, match="not found"):
        client.generate("non-existent-model", "Hello")

def test_client_list_models():
    server = MockOllamaServer()
    client = OllamaClient(server)
    models = client.list_models()
    assert len(models) >= 2
    assert "llama3:8b" in models

Troubleshooting

Troubleshooting

Ensure Modelfile parser strips quotation marks when extracting system prompts.

Security notes

Security

Runs completely in-memory using Python data structures. Zero external socket or port collisions during automated test runs.