LLMs and Generative AI › Customizing and Running Models › Day 277
Hands-on lab — Day 277: Running Local Models with Ollama
- ← Back to the Day 277 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/llms-and-generative-ai/day-277-running-local-models-with-ollama/
Commands
Setup
pip install -r requirements/requirements.txt Run
python examples/ollama_client.py Test
bash tests/run_tests.sh File tree
examples/ollama_client.py examples/test_ollama_client.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/ollama_client.py starter/test_ollama_client.py tests/run_tests.sh tests/test_ollama_client.py troubleshooting.md
Lab README
Day 277 Lab: Running Local Models with Ollama
Lesson
- Lesson title: Running Local Models with Ollama
- Day number: 277 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-277-running-local-models-with-ollama
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-277-running-local-models-with-ollamawhen the site is running.
Purpose
Build an Ollama Modelfile generator, mock streaming REST server, and client library in Python to simulate local LLM lifecycle operations and NDJSON token streaming.
Learning objectives
- Generate declarative Modelfiles with
FROM,PARAMETER, andSYSTEMblocks. - Simulate Ollama REST API endpoints (
/api/tags,/api/generate). - Parse chunked NDJSON streaming responses in Python.
Prerequisites
- Python 3.10+ installed
- pytest installed
Supported operating systems
- macOS, Linux, Windows WSL2
Hardware requirements
- Standard CPU, 512MB RAM
Required software
- Python 3.10+, pytest
Free and open-source options
- Python Standard Library
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/ollama_client.py: Starter implementationexamples/ollama_client.py: Verified reference implementationtests/test_ollama_client.py: Test suiteexpected-output/: Captured execution logs
How to run
python3 examples/ollama_client.py
What the commands do
- Compiles Modelfile configurations and streams simulated tokens through the client.
Expected output
[MODELFILE] Compiled Modelfile for 'custom-sql'
[STREAMING] Emitted 6 tokens | Duration: 120ms
Validation steps
Run test runner:
bash tests/run_tests.sh
Tests
The test suite validates:
- Modelfile string generation and parsing round-trips
- Tags endpoint response formatting
- Streaming NDJSON chunk parsing
- Error raising when requested models are absent
Cleanup
rm -rf .pytest_cache __pycache__ .venv
Troubleshooting
Verify Modelfile parameter casting parses integers and floats accurately.
Security notes
Zero external network transmission. Runs entirely locally on CPU.
Extension exercises
Add support for simulating /api/chat multi-turn message history.
Navigation
Day number: 277 of 365
Expected output
FIELDS.md
Field definitions for Ollama client. examples-run.txt
Example execution complete.
measured-values.txt
Models_Count=2
Eval_Count=6
Total_Duration_MS=120
run-output.txt
[MODELFILE] Compiled Modelfile for 'custom-sql'
[STREAMING] Emitted 6 tokens | Duration: 120ms
starter-run.txt
Starter execution complete.
test-output.txt
test-run.txt
test_ollama_client.py ..... [100%]
5 passed in 0.04s
Source files
examples/ollama_client.py (4686 bytes)
# Ollama Client, Modelfile Generator, and Local Mock REST Server
import json
from typing import Dict, List, Any, Optional, Iterator
class ModelfileGenerator:
"""Generates and validates declarative Ollama Modelfiles."""
@staticmethod
def build_modelfile(
from_model: str,
system_prompt: Optional[str] = None,
template: Optional[str] = None,
parameters: Optional[Dict[str, Any]] = None
) -> str:
lines = [f"FROM {from_model}"]
if parameters:
for key, val in sorted(parameters.items()):
lines.append(f"PARAMETER {key} {val}")
if system_prompt:
cleaned = system_prompt.strip()
lines.append(f'SYSTEM """{cleaned}"""')
if template:
cleaned_tpl = template.strip()
lines.append(f'TEMPLATE """{cleaned_tpl}"""')
return "\n".join(lines) + "\n"
@staticmethod
def parse_modelfile(content: str) -> Dict[str, Any]:
"""Parses a Modelfile string into structured configuration."""
config: Dict[str, Any] = {"parameters": {}}
for line in content.splitlines():
line = line.strip()
if not line or line.startswith("#"):
continue
if line.startswith("FROM "):
config["from"] = line[5:].strip()
elif line.startswith("PARAMETER "):
parts = line[10:].split(maxsplit=1)
if len(parts) == 2:
k, v = parts
# Cast types
if v.isdigit(): config["parameters"][k] = int(v)
else:
try: config["parameters"][k] = float(v)
except ValueError: config["parameters"][k] = v
elif line.startswith("SYSTEM "):
config["system"] = line[7:].strip('"')
elif line.startswith("TEMPLATE "):
config["template"] = line[9:].strip('"')
return config
class MockOllamaServer:
"""Simulates Ollama HTTP REST API streaming responses locally."""
def __init__(self):
self.registered_models = {
"llama3:8b": {"size": 4700000000, "family": "llama"},
"custom-sql:latest": {"size": 4900000000, "family": "llama"}
}
def handle_tags_request(self) -> Dict[str, Any]:
"""Simulates GET /api/tags"""
models = []
for name, meta in self.registered_models.items():
models.append({
"name": name,
"size": meta["size"],
"details": {"family": meta["family"], "parameter_size": "8B", "quantization_level": "Q4_0"}
})
return {"models": models}
def handle_generate_stream(self, model: str, prompt: str) -> Iterator[str]:
"""Simulates POST /api/generate NDJSON streaming."""
if model not in self.registered_models:
yield json.dumps({"error": f"model '{model}' not found"}) + "\n"
return
mock_words = f"Response to '{prompt[:20]}': SELECT * FROM database;".split()
for word in mock_words:
chunk = {
"model": model,
"response": word + " ",
"done": False
}
yield json.dumps(chunk) + "\n"
# Final chunk
final_chunk = {
"model": model,
"response": "",
"done": True,
"total_duration": 120000000,
"eval_count": len(mock_words),
"eval_duration": 100000000
}
yield json.dumps(final_chunk) + "\n"
class OllamaClient:
"""Client library for communicating with Ollama REST API."""
def __init__(self, server: MockOllamaServer):
self.server = server
def list_models(self) -> List[str]:
data = self.server.handle_tags_request()
return [m["name"] for m in data["models"]]
def generate(self, model: str, prompt: str) -> Dict[str, Any]:
full_text = []
eval_count = 0
total_duration = 0
for line in self.server.handle_generate_stream(model, prompt):
chunk = json.loads(line.strip())
if "error" in chunk:
raise ValueError(chunk["error"])
full_text.append(chunk.get("response", ""))
if chunk.get("done", False):
eval_count = chunk.get("eval_count", 0)
total_duration = chunk.get("total_duration", 0)
return {
"response": "".join(full_text).strip(),
"eval_count": eval_count,
"total_duration_ms": total_duration / 1_000_000.0
}
examples/test_ollama_client.py (1674 bytes)
import pytest
import json
from ollama_client import ModelfileGenerator, MockOllamaServer, OllamaClient
def test_modelfile_builder_and_parser():
modelfile = ModelfileGenerator.build_modelfile(
from_model="./custom_model.gguf",
system_prompt="You are a SQL expert.",
parameters={"temperature": 0.2, "num_ctx": 4096}
)
assert "FROM ./custom_model.gguf" in modelfile
assert "PARAMETER num_ctx 4096" in modelfile
assert 'SYSTEM """You are a SQL expert."""' in modelfile
parsed = ModelfileGenerator.parse_modelfile(modelfile)
assert parsed["from"] == "./custom_model.gguf"
assert parsed["parameters"]["temperature"] == 0.2
assert parsed["parameters"]["num_ctx"] == 4096
def test_mock_server_tags_endpoint():
server = MockOllamaServer()
tags = server.handle_tags_request()
model_names = [m["name"] for m in tags["models"]]
assert "llama3:8b" in model_names
assert "custom-sql:latest" in model_names
def test_streaming_generation():
server = MockOllamaServer()
client = OllamaClient(server)
res = client.generate("llama3:8b", "Write a query")
assert "SELECT * FROM database;" in res["response"]
assert res["eval_count"] > 0
assert res["total_duration_ms"] == 120.0
def test_model_not_found_error():
server = MockOllamaServer()
client = OllamaClient(server)
with pytest.raises(ValueError, match="not found"):
client.generate("non-existent-model", "Hello")
def test_client_list_models():
server = MockOllamaServer()
client = OllamaClient(server)
models = client.list_models()
assert len(models) >= 2
assert "llama3:8b" in models
metadata.yml (400 bytes)
lesson_id: D277
day: 277
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/ollama_client.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=8.0.0
starter/ollama_client.py (788 bytes)
# Starter: Ollama Client and Modelfile Generator
from typing import Dict, List, Any, Optional
class ModelfileGenerator:
@staticmethod
def build_modelfile(
from_model: str,
system_prompt: Optional[str] = None,
template: Optional[str] = None,
parameters: Optional[Dict[str, Any]] = None
) -> str:
return f"FROM {from_model}\n"
class MockOllamaServer:
def handle_tags_request(self) -> Dict[str, Any]:
return {"models": []}
class OllamaClient:
def __init__(self, server: MockOllamaServer):
self.server = server
def list_models(self) -> List[str]:
return []
def generate(self, model: str, prompt: str) -> Dict[str, Any]:
return {"response": "", "eval_count": 0, "total_duration_ms": 0.0}
starter/test_ollama_client.py (1674 bytes)
import pytest
import json
from ollama_client import ModelfileGenerator, MockOllamaServer, OllamaClient
def test_modelfile_builder_and_parser():
modelfile = ModelfileGenerator.build_modelfile(
from_model="./custom_model.gguf",
system_prompt="You are a SQL expert.",
parameters={"temperature": 0.2, "num_ctx": 4096}
)
assert "FROM ./custom_model.gguf" in modelfile
assert "PARAMETER num_ctx 4096" in modelfile
assert 'SYSTEM """You are a SQL expert."""' in modelfile
parsed = ModelfileGenerator.parse_modelfile(modelfile)
assert parsed["from"] == "./custom_model.gguf"
assert parsed["parameters"]["temperature"] == 0.2
assert parsed["parameters"]["num_ctx"] == 4096
def test_mock_server_tags_endpoint():
server = MockOllamaServer()
tags = server.handle_tags_request()
model_names = [m["name"] for m in tags["models"]]
assert "llama3:8b" in model_names
assert "custom-sql:latest" in model_names
def test_streaming_generation():
server = MockOllamaServer()
client = OllamaClient(server)
res = client.generate("llama3:8b", "Write a query")
assert "SELECT * FROM database;" in res["response"]
assert res["eval_count"] > 0
assert res["total_duration_ms"] == 120.0
def test_model_not_found_error():
server = MockOllamaServer()
client = OllamaClient(server)
with pytest.raises(ValueError, match="not found"):
client.generate("non-existent-model", "Hello")
def test_client_list_models():
server = MockOllamaServer()
client = OllamaClient(server)
models = client.list_models()
assert len(models) >= 2
assert "llama3:8b" in models
tests/run_tests.sh (122 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
PYTEST="${PYTEST:-pytest}"
$PYTEST tests/test_ollama_client.py -v
tests/test_ollama_client.py (1674 bytes)
import pytest
import json
from ollama_client import ModelfileGenerator, MockOllamaServer, OllamaClient
def test_modelfile_builder_and_parser():
modelfile = ModelfileGenerator.build_modelfile(
from_model="./custom_model.gguf",
system_prompt="You are a SQL expert.",
parameters={"temperature": 0.2, "num_ctx": 4096}
)
assert "FROM ./custom_model.gguf" in modelfile
assert "PARAMETER num_ctx 4096" in modelfile
assert 'SYSTEM """You are a SQL expert."""' in modelfile
parsed = ModelfileGenerator.parse_modelfile(modelfile)
assert parsed["from"] == "./custom_model.gguf"
assert parsed["parameters"]["temperature"] == 0.2
assert parsed["parameters"]["num_ctx"] == 4096
def test_mock_server_tags_endpoint():
server = MockOllamaServer()
tags = server.handle_tags_request()
model_names = [m["name"] for m in tags["models"]]
assert "llama3:8b" in model_names
assert "custom-sql:latest" in model_names
def test_streaming_generation():
server = MockOllamaServer()
client = OllamaClient(server)
res = client.generate("llama3:8b", "Write a query")
assert "SELECT * FROM database;" in res["response"]
assert res["eval_count"] > 0
assert res["total_duration_ms"] == 120.0
def test_model_not_found_error():
server = MockOllamaServer()
client = OllamaClient(server)
with pytest.raises(ValueError, match="not found"):
client.generate("non-existent-model", "Hello")
def test_client_list_models():
server = MockOllamaServer()
client = OllamaClient(server)
models = client.list_models()
assert len(models) >= 2
assert "llama3:8b" in models
Troubleshooting
Troubleshooting
Ensure Modelfile parser strips quotation marks when extracting system prompts.
Security notes
Security
Runs completely in-memory using Python data structures. Zero external socket or port collisions during automated test runs.