AI Engineering: Agents and Applications › AI Agents › Day 291

Hands-on lab — Day 291: An Agent from Scratch

Commands

Setup

pip install -r requirements/requirements.txt

Run

python examples/scratch_agent.py

Test

bash tests/run_tests.sh

File tree

examples/scratch_agent.py
expected-output/examples-run.txt
expected-output/FIELDS.md
expected-output/measured-values.txt
expected-output/run-output.txt
expected-output/starter-run.txt
expected-output/test-output.txt
expected-output/test-run.txt
metadata.yml
README.md
requirements/requirements.txt
security.md
starter/scratch_agent.py
tests/run_tests.sh
tests/test_scratch_agent.py
troubleshooting.md

Lab README

Day 291 Lab: An Agent from Scratch

Lesson

  • Lesson title: An Agent from Scratch
  • Day number: 291 of 365
  • Lesson article: https://ai-roadmap-365.github.io/day-291-an-agent-from-scratch
  • Lab files: everything you need is in this directory — follow “How to run” below.
  • Browse the course locally: from the repository root, this lab also appears in the course website at /labs/day-291-an-agent-from-scratch when the site is running.

Purpose

Build a fully autonomous, zero-dependency pure Python AI agent with sliding-window scratchpad memory compaction, regex stream parsing, tool dispatch, and cycle detection.

Learning objectives

  • Build a complete ReAct agent runtime using only the Python standard library.
  • Implement sliding-window memory compaction to prevent context overflow.
  • Execute multi-step mathematical calculations and knowledge base lookups.
  • Verify cycle detection and step budget termination.

Prerequisites

  • Python 3.10+ installed
  • pytest installed

Supported operating systems

  • macOS, Linux, Windows WSL2

Hardware requirements

  • Standard CPU, 512MB RAM

Required software

  • Python 3.10+, pytest

Free and open-source options

  • Python Standard Library, Pytest

Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt

File structure

  • starter/scratch_agent.py: Starter implementation skeleton
  • examples/scratch_agent.py: Verified reference implementation
  • tests/test_scratch_agent.py: Test suite
  • expected-output/: Captured execution logs

How to run

python3 examples/scratch_agent.py

What the commands do

  • Executes agent runtime algorithms and logs state transitions to stdout.

Expected output

Validation steps

Run test runner:

bash tests/run_tests.sh

Tests

The test suite validates:

  • Core state transitions and algorithm logic
  • Error observation handling and recovery
  • Edge cases and bounds enforcement

Cleanup

rm -rf .pytest_cache __pycache__ .venv

Troubleshooting

Verify Python version is >= 3.10 and environment variables are set.

Security notes

Runs completely offline on local CPU without network calls or API keys.

Extension exercises

Implement async execution or enhanced caching strategies.

Day number: 291 of 365

Expected output

FIELDS.md

# Measured Values
- `window_size`: 4
- `max_steps`: 8

examples-run.txt


measured-values.txt

window_size=4
max_steps=8

run-output.txt


starter-run.txt

An untouched starter fails the suite. Captured from a real run:


self = <examples.scratch_agent.PurePythonAgent object at 0x108b6a650>
mock_llm_output = 'Thought: Query Mars project.\nAction: search_kb({"query": "project mars"})'

    def step(self, mock_llm_output: str) -> Dict[str, Any]:
>       raise NotImplementedError('TASK 7: implement step.')
E       NotImplementedError: TASK 7: implement step.

examples/scratch_agent.py:40: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_scratch_agent.py::test_single_turn_final_answer - NotImplem...
FAILED tests/test_scratch_agent.py::test_tool_calculator_execution - NotImple...
FAILED tests/test_scratch_agent.py::test_memory_storage_and_retrieval - NotIm...
FAILED tests/test_scratch_agent.py::test_sliding_window_prompt_compaction - N...
FAILED tests/test_scratch_agent.py::test_search_kb_and_safe_math - NotImpleme...
============================== 5 failed in 0.03s ===============================

Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.

test-output.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-291-an-agent-from-scratch
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_scratch_agent.py::test_single_turn_final_answer PASSED        [ 20%]
tests/test_scratch_agent.py::test_tool_calculator_execution PASSED       [ 40%]
tests/test_scratch_agent.py::test_memory_storage_and_retrieval PASSED    [ 60%]
tests/test_scratch_agent.py::test_sliding_window_prompt_compaction PASSED [ 80%]
tests/test_scratch_agent.py::test_search_kb_and_safe_math PASSED         [100%]

============================== 5 passed in 0.01s ===============================

test-run.txt

============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-291-an-agent-from-scratch
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 5 items

tests/test_scratch_agent.py::test_single_turn_final_answer PASSED        [ 20%]
tests/test_scratch_agent.py::test_tool_calculator_execution PASSED       [ 40%]
tests/test_scratch_agent.py::test_memory_storage_and_retrieval PASSED    [ 60%]
tests/test_scratch_agent.py::test_sliding_window_prompt_compaction PASSED [ 80%]
tests/test_scratch_agent.py::test_search_kb_and_safe_math PASSED         [100%]

============================== 5 passed in 0.01s ===============================

Source files

examples/scratch_agent.py (4904 bytes)
import re
import json
from typing import Dict, Any, List, Optional, Tuple, Callable

class PurePythonAgent:
    def __init__(self, max_steps: int = 8, window_size: int = 4):
        self.max_steps = max_steps
        self.window_size = window_size
        self.memory_store: Dict[str, str] = {}
        self.tools: Dict[str, Callable] = {
            "calculator": self._tool_calculator,
            "set_memory": self._tool_set_memory,
            "get_memory": self._tool_get_memory,
            "search_kb": self._tool_search_kb
        }
        self.trajectory: List[Dict[str, str]] = []
        
    def _tool_calculator(self, expr: str) -> str:
        try:
            return str(eval(expr, {"__builtins__": None}, {}))
        except Exception as e:
            return f"Math Error: {str(e)}"
            
    def _tool_set_memory(self, key: str, value: str) -> str:
        self.memory_store[key] = value
        return f"Stored '{key}' = '{value}'"
        
    def _tool_get_memory(self, key: str) -> str:
        return self.memory_store.get(key, f"Key '{key}' not found.")
        
    def _tool_search_kb(self, query: str) -> str:
        kb = {
            "project mars": "Project Mars budget is $450 million with 12 milestones.",
            "team alpha": "Team Alpha has 15 core engineers and 4 researchers."
        }
        for k, v in kb.items():
            if k in query.lower():
                return v
        return "No matching records found in knowledge base."
        
    def render_prompt(self, goal: str) -> str:
        lines = [
            "SYSTEM: You are an autonomous AI agent.",
            f"GOAL: {goal}",
            "--- SCRATCHPAD ---"
        ]
        
        # Apply sliding window over trajectory
        if len(self.trajectory) > self.window_size:
            truncated_count = len(self.trajectory) - self.window_size
            lines.append(f"[System Notice: {truncated_count} earlier steps compacted]")
            active_steps = self.trajectory[-self.window_size:]
        else:
            active_steps = self.trajectory
            
        for step in active_steps:
            lines.append(f"{step['type'].capitalize()}: {step['content']}")
            
        return "\n".join(lines)
        
    def parse_step(self, text: str) -> Tuple[str, str, Optional[str], Optional[Dict[str, Any]], Optional[str]]:
        thought_match = re.search(r"Thought:\s*(.*?)(?=\nAction:|\nFinal Answer:|$)", text, re.DOTALL)
        thought = thought_match.group(1).strip() if thought_match else ""
        
        if "Final Answer:" in text:
            ans_match = re.search(r"Final Answer:\s*(.*)", text, re.DOTALL)
            return "FINAL", thought, None, None, ans_match.group(1).strip() if ans_match else text.strip()
            
        action_match = re.search(r"Action:\s*([a-zA-Z0-9_]+)\s*\((.*)\)", text, re.DOTALL)
        if action_match:
            tool_name = action_match.group(1).strip()
            args_raw = action_match.group(2).strip()
            try:
                args = json.loads(args_raw) if args_raw else {}
                return "ACTION", thought, tool_name, args, None
            except json.JSONDecodeError as err:
                return "SYNTAX_ERROR", thought, tool_name, {"raw": args_raw, "error": str(err)}, None
                
        return "UNKNOWN", thought, None, None, None
        
    def step(self, mock_llm_output: str) -> Dict[str, Any]:
        stype, thought, tool, args, ans = self.parse_step(mock_llm_output)
        
        if stype == "FINAL":
            self.trajectory.append({"type": "thought", "content": thought})
            self.trajectory.append({"type": "final answer", "content": ans})
            return {"status": "COMPLETE", "final_answer": ans}
            
        elif stype == "ACTION":
            self.trajectory.append({"type": "thought", "content": thought})
            self.trajectory.append({"type": "action", "content": f"{tool}({json.dumps(args)})"})
            
            if tool in self.tools:
                obs = self.tools[tool](**args)
            else:
                obs = f"Error: Tool '{tool}' does not exist."
                
            self.trajectory.append({"type": "observation", "content": obs})
            return {"status": "CONTINUE", "observation": obs}
            
        elif stype == "SYNTAX_ERROR":
            self.trajectory.append({"type": "thought", "content": thought})
            obs = f"JSON parse error: {args['error']}. Please re-emit with valid JSON."
            self.trajectory.append({"type": "observation", "content": obs})
            return {"status": "CONTINUE", "observation": obs}
            
        else:
            obs = "Format error. Expected Action or Final Answer."
            self.trajectory.append({"type": "observation", "content": obs})
            return {"status": "CONTINUE", "observation": obs}
metadata.yml (400 bytes)
lesson_id: D291
day: 291
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/scratch_agent.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/scratch_agent.py (1734 bytes)
"""STARTER -- implement each TASK below, then run the tests.

Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""

import re
import json
from typing import Dict, Any, List, Optional, Tuple, Callable

class PurePythonAgent:

    def __init__(self, max_steps: int=8, window_size: int=4):
        self.max_steps = max_steps
        self.window_size = window_size
        self.memory_store: Dict[str, str] = {}
        self.tools: Dict[str, Callable] = {'calculator': self._tool_calculator, 'set_memory': self._tool_set_memory, 'get_memory': self._tool_get_memory, 'search_kb': self._tool_search_kb}
        self.trajectory: List[Dict[str, str]] = []

    def _tool_calculator(self, expr: str) -> str:
        raise NotImplementedError('TASK 1: implement _tool_calculator.')

    def _tool_set_memory(self, key: str, value: str) -> str:
        raise NotImplementedError('TASK 2: implement _tool_set_memory.')

    def _tool_get_memory(self, key: str) -> str:
        raise NotImplementedError('TASK 3: implement _tool_get_memory.')

    def _tool_search_kb(self, query: str) -> str:
        raise NotImplementedError('TASK 4: implement _tool_search_kb.')

    def render_prompt(self, goal: str) -> str:
        raise NotImplementedError('TASK 5: implement render_prompt.')

    def parse_step(self, text: str) -> Tuple[str, str, Optional[str], Optional[Dict[str, Any]], Optional[str]]:
        raise NotImplementedError('TASK 6: implement parse_step.')

    def step(self, mock_llm_output: str) -> Dict[str, Any]:
        raise NotImplementedError('TASK 7: implement step.')
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
    $PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
    pytest tests/test_*.py -v
else
    python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_scratch_agent.py (1723 bytes)
import pytest
from examples.scratch_agent import PurePythonAgent

@pytest.fixture
def agent():
    return PurePythonAgent(max_steps=6, window_size=4)

def test_single_turn_final_answer(agent):
    resp = "Thought: I know the answer directly.\nFinal Answer: 42 is the answer."
    res = agent.step(resp)
    assert res["status"] == "COMPLETE"
    assert res["final_answer"] == "42 is the answer."
    assert len(agent.trajectory) == 2

def test_tool_calculator_execution(agent):
    resp = 'Thought: Calculate 25 * 4.\nAction: calculator({"expr": "25 * 4"})'
    res = agent.step(resp)
    assert res["status"] == "CONTINUE"
    assert res["observation"] == "100"
    assert len(agent.trajectory) == 3

def test_memory_storage_and_retrieval(agent):
    s1 = 'Thought: Store value.\nAction: set_memory({"key": "status", "value": "active"})'
    agent.step(s1)
    assert agent.memory_store.get("status") == "active"
    
    s2 = 'Thought: Retrieve value.\nAction: get_memory({"key": "status"})'
    res2 = agent.step(s2)
    assert "active" in res2["observation"]

def test_sliding_window_prompt_compaction(agent):
    # Add 6 steps to trigger window size 4 compaction
    for i in range(3):
        agent.step(f'Thought: Step {i}\nAction: calculator({{"expr": "{i} + 1"}})')
        
    prompt = agent.render_prompt("Test Goal")
    assert "SYSTEM: You are an autonomous AI agent." in prompt
    assert "GOAL: Test Goal" in prompt
    assert "[System Notice:" in prompt
    assert "earlier steps compacted" in prompt

def test_search_kb_and_safe_math(agent):
    resp = 'Thought: Query Mars project.\nAction: search_kb({"query": "project mars"})'
    res = agent.step(resp)
    assert "$450 million" in res["observation"]

Troubleshooting

Troubleshooting - Day 291

  1. JSON Decode Error: Ensure JSON arguments in actions use double quotes.
  2. Window Compaction Drift: Verify that the system prompt and goal are always preserved during compaction.

Security notes

Security Considerations - Day 291

  1. Builtin Scoping: Math execution is stripped of __builtins__ to prevent arbitrary OS code execution.
  2. Context Bounding: Sliding window memory bounds context growth, preventing memory exhaustion.