AI Engineering: Agents and Applications β€Ί AI Agents β€Ί Day 291

Day 291: An Agent from Scratch

Day 291 of 365 β€” An Agent from Scratch

Build a fully autonomous, production-ready AI agent from scratch in zero-dependency pure Python: implement stateful memory, regex stream parsing, tool dispatch, sliding-window compaction, and cycle detection.

Course
AI Engineering: Agents and Applications
Category
AI Agents
Reading time
β‰ˆ 35 min
Practical time
β‰ˆ 50 min
Lesson duration
1h 25m
Last verified
2026-08-30

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/ai-engineering/day-291-an-agent-from-scratch

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path β€” section / subsection / week / day:
    cd labs/sections/ai-engineering/day-291-an-agent-from-scratch
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work β€” read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

When software engineers first explore agent development, they are frequently directed toward heavy open-source frameworks featuring hundreds of thousands of lines of abstract wrapper classes, complex callback hooks, and sprawling dependency graphs. While frameworks offer quick five-minute starter templates, they create immense technical debt when deployed to high-throughput production environments: subtle memory leaks in asynchronous execution loops are difficult to trace, token usage is obscured behind multiple layers of middleware, and debugging a malformed prompt requires stepping through dozens of nested library files.

Building a complete, production-grade autonomous agent from scratch using vanilla Python and standard library primitives is the ultimate test of engineering mastery. It strips away all magical abstractions, exposing the raw mechanics of agentic computation: prompt assembly, token stream serialization, deterministic action parsing, sandbox execution, sliding window context memory, and stopping predicates.

By building an agent with zero external dependencies, you gain total transparency over your AI systems, achieve sub-millisecond runtime overhead, and develop the architectural clarity needed to evaluate when third-party frameworks are genuinely necessary.

Zero Dependency Agent Scratchpad Architecture illustrating System Prompt, Dynamic Scratchpad Buffer, Tool Registry, and Execution Dispatcher

The idea in plain language

Diagram: a pre-built enclosed vehicle compared with one assembled from components

Diagram: the six components of a zero-dependency agent runtime

Think of building an agent from scratch like building an electric bicycle from individual components versus buying an enclosed pre-built scooter:

In software engineering, a zero-dependency agent is transparent, lightweight, easily unit-tested, and virtually indestructible.

Historical background

Diagram: the swing from monolithic agent frameworks back to explicit loops

The evolution of agent implementations reflects a classic software engineering pendulum between monolithic frameworks and lightweight micro-architectures:

1. The Heavy Framework Gold Rush (2022–2023)

Following the release of ChatGPT in late 2022, open-source agent frameworks exploded in popularity (such as early LangChain and AutoGPT). These libraries attempted to be universal solutions, wrapping every vector database, model provider, and scraper into monolithic class hierarchies. However, developers quickly encountered severe production friction: breaking API changes across minor releases, undocumented prompt mutations, and excessive latency overhead.

2. The Return to Simplicity (2024–Present)

In 2024, leading AI research labs and production engineering teams advocated for a return to lightweight, explicit agent architectures. Anthropic published Building Effective Agents, demonstrating that the most reliable enterprise systems are not complex multi-layered agent frameworks, but simple, composable Python loops utilizing explicit prompt formatting and native tool calling APIs.

What it is β€” and what it is not

Let us define the engineering parameters of a zero-dependency agent:

What a Zero-Dependency Agent Is:

What a Zero-Dependency Agent Is Not:

Diagram: what a from-scratch agent includes compared with three misconceptions

+-------------------------------------------------------------------------------+
|                      FRAMEWORK OVERHEAD COMPARISON                            |
+---------------------+-----------------------+---------------------------------+
| Metric              | Monolithic Framework  | Zero-Dependency Scratch Build   |
+---------------------+-----------------------+---------------------------------+
| Dependency Tree     | 40+ external packages | 0 third-party packages (stdlib) |
| Runtime Overhead    | 15 - 50 ms per step   | < 0.2 ms per step               |
| Prompt Inspection   | Obscured in callbacks | 100% Explicit String Variable   |
| Debuggability       | Deep stack traces     | Single-file stepping            |
| Cold-Start Latency  | ~450 ms (import cost) | ~2 ms (instantaneous startup)   |
+---------------------+-----------------------+---------------------------------+

Why it was created and what problems it solves

Building an agent from scratch solves four universal enterprise engineering challenges:

1. Eliminating Vulnerability Supply-Chain Risks

Enterprise security teams enforce strict vulnerability scanning (CVE checks). Monolithic frameworks frequently pull in dozens of transitive dependencies (e.g. outdated XML parsers, unmaintained scraping utilities), triggering security audit blocks. A zero-dependency core contains zero third-party CVE exposure.

2. Total Token Transparency and Cost Control

Monolithic frameworks often inject hidden system prompts, preamble boilerplate, and verbose JSON schemas behind the scenes, inflating input token consumption by 30% to 50%. A custom agent builder dictates every single token in the context window.

3. Predictable Exception and Error Recovery

When a third-party framework throws an obscure OutputParserException or AgentStoppedUnexpectedlyError, handling it gracefully in production is painful. In a scratch build, you define exact exception classes and custom observation formatting.

4. Deterministic Offline Testing

You can mock the LLM gateway with simple rule-based Python functions, running hundreds of comprehensive unit tests in CI/CD pipelines in less than two seconds without external network access or paid API keys.

How it works

Diagram: the four engineering arguments for building the agent loop yourself

Let us inspect the architecture of a production zero-dependency agent:

+─────────────────────────────────────────────────────────────────────────────+
|                             CustomAgentRuntime                              |
+─────────────────────────────────────────────────────────────────────────────+
|  - System Prompt Generator (Rules + Tool JSON Schemas)                      |
|  - Memory Buffer (Sliding Window Trajectory)                               |
|  - Step Dispatcher (Regex / JSON Action Extraction)                         |
|  - Sandbox Tool Registry (Math, Memory, Search)                             |
|  - Telemetry Logger (Step counts, token estimates, latency)                 |
+─────────────────────────────────────────────────────────────────────────────+
                                       β”‚
                         [Prompt Stream Formatting]
                                       β”‚
                                       β–Ό
                     [LLM Gateway / Mock Inference Engine]
                                       β”‚
                                       β–Ό
                     [Action Parser & Parameter Coercer]
                                       β”‚
                                       β–Ό
                     [Isolated Tool Sandbox Execution]
                                       β”‚
                                       β–Ό
                     [Sliding Window Context Compactor]
                                       β”‚
                                       β–Ό
                   (Loop Check: Goal Complete or Step > Max)

1. The Trajectory Step Dataclass

Every event in the agent’s lifetime is captured as an immutable dataclass:

from dataclasses import dataclass, field
from typing import Optional, Dict, Any

@dataclass
class TrajectoryStep:
    step_number: int
    step_type: str # "thought", "action", "observation", "final_answer", "error"
    content: str
    metadata: Dict[str, Any] = field(default_factory=dict)

2. Context Window Compaction via Sliding Window

As the agent iterates, the prompt grows. To prevent exceeding model context limits, the runtime preserves the initial system prompt and goal, compacts intermediate turns into structured bullet summaries, and retains the most recent $K$ turns in full raw detail.

Diagram: how a sliding window compacts an agent's scratchpad

Agent Short Term Memory Sliding Window diagram showing system prompt preservation, middle step truncation, and recent turn retention

An everyday analogy

Diagram: a disposable digital watch compared with a mechanical chronometer

Think of building an agent from scratch like a master watchmaker crafting a mechanical chronometer:

When you build your agent from scratch, you are the master watchmaker of your AI architecture.

Deep Dive: Production Telemetry & Secure State Serialization

Diagram: in-process telemetry and JSON state serialisation for agent resumption

When designing custom zero-dependency agent runtimes for high-throughput enterprise deployments, telemetry and observability must be built directly into the core execution loop. Because standard logging frameworks introduce non-trivial I/O locking, production agents utilize in-memory circular ring buffers that capture nanosecond-resolution timestamps, token consumption estimates, and exact tool invocation payloads.

+-------------------------------------------------------------------------------+
|                       AGENT TELEMETRY METRIC SPECIFICATION                    |
+-------------------+-----------------------+--------------------+--------------+
| Metric Name       | Measurement Unit      | Sampling Frequency | Alert Threshold|
+-------------------+-----------------------+--------------------+--------------+
| Step Latency      | Milliseconds (ms)     | Every ReAct Turn   | > 4,000 ms   |
| Scratchpad Size   | Estimated Tokens      | Every ReAct Turn   | > 8,000 tok  |
| Tool Execution Err| Error Count / Rate    | Per Tool Call      | > 2 retries  |
| Memory Compaction | Compression Ratio (%) | Upon Truncation    | < 50% ratio  |
+-------------------+-----------------------+--------------------+--------------+

Furthermore, state persistence across distributed worker nodes requires robust serialization protocols. While standard Python pickle introduces critical remote code execution vulnerabilities, a pure-Python agent serializes its complete state into canonical JSON schemas or SQLite WAL databases. This enables instant worker failover: if a container process crashes during step 5 of an 8-step migration task, a replacement worker node inspects the database, deserializes the JSON state history, and seamlessly resumes execution without repeating earlier tool mutations.

Examples in practice

Let us inspect a complete, zero-dependency autonomous Calculator, Memory, and Search Agent implemented in pure standard-library Python:

import re
import json
from typing import Dict, Any, List, Optional, Callable

class PurePythonAgent:
    def __init__(self, max_steps: int = 8, window_size: int = 4):
        self.max_steps = max_steps
        self.window_size = window_size
        self.memory_store: Dict[str, str] = {}
        self.tools: Dict[str, Callable] = {
            "calculator": self._tool_calculator,
            "set_memory": self._tool_set_memory,
            "get_memory": self._tool_get_memory,
            "search_kb": self._tool_search_kb
        }
        self.trajectory: List[Dict[str, str]] = []
        
    def _tool_calculator(self, expr: str) -> str:
        try:
            return str(eval(expr, {"__builtins__": None}, {}))
        except Exception as e:
            return f"Math Error: {str(e)}"
            
    def _tool_set_memory(self, key: str, value: str) -> str:
        self.memory_store[key] = value
        return f"Stored '{key}' = '{value}' in persistent memory."
        
    def _tool_get_memory(self, key: str) -> str:
        return self.memory_store.get(key, f"Key '{key}' not found.")
        
    def _tool_search_kb(self, query: str) -> str:
        kb = {
            "project mars": "Project Mars budget is $450 million with 12 milestones.",
            "team alpha": "Team Alpha has 15 core engineers and 4 researchers."
        }
        for k, v in kb.items():
            if k in query.lower():
                return v
        return "No matching records found in knowledge base."
        
    def render_prompt(self, goal: str) -> str:
        lines = [
            "SYSTEM: You are an autonomous AI agent. Solve the user goal using ReAct steps:",
            "Thought: reasoning",
            "Action: tool_name({"param": "value"})",
            "Observation: result",
            "Final Answer: final response",
            f"GOAL: {goal}",
            "--- SCRATCHPAD ---"
        ]
        
        # Apply sliding window over trajectory
        if len(self.trajectory) > self.window_size:
            truncated_count = len(self.trajectory) - self.window_size
            lines.append(f"[System Notice: {truncated_count} earlier steps compacted]")
            active_steps = self.trajectory[-self.window_size:]
        else:
            active_steps = self.trajectory
            
        for step in active_steps:
            lines.append(f"{step['type'].capitalize()}: {step['content']}")
            
        return "
".join(lines)
        
    def parse_step(self, text: str) -> Tuple[str, str, Optional[str], Optional[Dict[str, Any]], Optional[str]]:
        thought_match = re.search(r"Thought:\s*(.*?)(?=
Action:|
Final Answer:|$)", text, re.DOTALL)
        thought = thought_match.group(1).strip() if thought_match else ""
        
        if "Final Answer:" in text:
            ans_match = re.search(r"Final Answer:\s*(.*)", text, re.DOTALL)
            return "FINAL", thought, None, None, ans_match.group(1).strip() if ans_match else text.strip()
            
        action_match = re.search(r"Action:\s*([a-zA-Z0-9_]+)\s*\((.*)\)", text, re.DOTALL)
        if action_match:
            tool_name = action_match.group(1).strip()
            args_raw = action_match.group(2).strip()
            try:
                args = json.loads(args_raw) if args_raw else {}
                return "ACTION", thought, tool_name, args, None
            except json.JSONDecodeError as err:
                return "SYNTAX_ERROR", thought, tool_name, {"raw": args_raw, "error": str(err)}, None
                
        return "UNKNOWN", thought, None, None, None
        
    def step(self, mock_llm_output: str) -> Dict[str, Any]:
        stype, thought, tool, args, ans = self.parse_step(mock_llm_output)
        
        if stype == "FINAL":
            self.trajectory.append({"type": "thought", "content": thought})
            self.trajectory.append({"type": "final answer", "content": ans})
            return {"status": "COMPLETE", "final_answer": ans}
            
        elif stype == "ACTION":
            self.trajectory.append({"type": "thought", "content": thought})
            self.trajectory.append({"type": "action", "content": f"{tool}({json.dumps(args)})"})
            
            if tool in self.tools:
                obs = self.tools[tool](**args)
            else:
                obs = f"Error: Tool '{tool}' does not exist."
                
            self.trajectory.append({"type": "observation", "content": obs})
            return {"status": "CONTINUE", "observation": obs}
            
        elif stype == "SYNTAX_ERROR":
            self.trajectory.append({"type": "thought", "content": thought})
            obs = f"JSON parse error: {args['error']}. Please re-emit with valid JSON."
            self.trajectory.append({"type": "observation", "content": obs})
            return {"status": "CONTINUE", "observation": obs}
            
        else:
            obs = "Format error. Expected Action or Final Answer."
            self.trajectory.append({"type": "observation", "content": obs})
            return {"status": "CONTINUE", "observation": obs}

Implications: security, privacy, performance, scalability, and cost

Diagram: the transparency, performance and testability properties of a scratch agent

1. Safe Builtin Scoping

Notice in the _tool_calculator method: eval(expr, {"__builtins__": None}, {}). Stripping Python builtins prevents users or malicious prompt injections from invoking __import__('os').system('rm -rf /').

Diagram: evaluating a model-supplied expression with and without builtins available

2. Sub-Millisecond Execution Overhead

Because this implementation avoids heavy async reflection, meta-programming wrappers, and complex logging cascades, the entire state transition and parsing executes in less than 150 microseconds (0.15\text( ms)), leaving 99.9% of the latency budget for model inference.

Alternatives: free, open source, and commercial

ParadigmControl LevelMaintenance CostEcosystem Integrations
Zero-Dependency Scratch Build100% Complete ControlLow (Single Python File)Custom (Hand-written)
LangGraph (Open Source)High (State Graphs)ModerateLarge community ecosystem
LlamaIndex WorkflowsEvent-Driven EventsModerateNative RAG index integration
OpenAI Assistants APIManaged Cloud ServerLow (Hosted API)OpenAI platform only
+-----------------------+-----------------------+-----------------------+
| Dimension             | Monolithic Framework  | Scratch Python Agent  |
+-----------------------+-----------------------+-----------------------+
| External Dependencies | 30 - 60 packages      | 0 (Pure Standard Lib) |
| Code Footprint        | > 100,000 LOC         | ~150 - 300 LOC        |
| Test Execution Speed  | ~3 - 10 seconds       | < 0.05 seconds        |
| Custom Memory Support | Must fit BaseMemory   | Any custom data struct|
| Production Portability| Python package envs   | Single copy-paste file|
+-----------------------+-----------------------+-----------------------+

When to use it β€” and when not to

Diagram: when a zero-dependency agent is the right choice

Choose a Zero-Dependency Scratch Agent when:

Do NOT use a Zero-Dependency Scratch Agent when:

The AI connection

Knowledge check

  1. What is the primary advantage of a zero-dependency agent architecture? Complete execution transparency, zero supply-chain security vulnerabilities, sub-millisecond runtime overhead, and simplified debugging.
  2. How does sliding window memory prevent context overflow? It retains the base system prompt and recent turns while compacting or summarizing intermediate turns.
  3. How do you safely evaluate mathematical expressions in Python? By passing {"__builtins__": None} to eval() to eliminate access to dangerous system modules.

Hands-on exercise

In this lab, you will implement an Autonomous Zero-Dependency Agent with Multi-Turn Memory and Sliding Window Compaction in Python. Your implementation will:

  1. Build a pure-Python agent runtime supporting Calculator, Key-Value Fact Store, and Knowledge Search tools.
  2. Implement robust ReAct token stream parsing with syntax error recovery.
  3. Enforce a sliding window context compactor that bounds prompt token growth across long-horizon trajectories.
  4. Execute simulated multi-step agent trajectories and verify 100% test passing against unit tests.

Expected output

============================= test session starts ==============================
collected 5 items

tests/test_scratch_agent.py .....                                        [100%]

============================== 5 passed in 0.04s ===============================
[SCRATCH AGENT] Goal: "Query Project Mars budget and calculate cost per milestone"
[TURN 1] THOUGHT: I need to search knowledge base for Project Mars.
         ACTION: search_kb({"query": "Project Mars"})
         OBSERVATION: "Project Mars budget is $450 million with 12 milestones."
[TURN 2] THOUGHT: Now calculating 450 / 12
         ACTION: calculator({"expr": "450 / 12"})
         OBSERVATION: "37.5"
[TURN 3] THOUGHT: Storing calculation in memory.
         ACTION: set_memory({"key": "mars_milestone_cost", "value": "$37.5M"})
         OBSERVATION: "Stored 'mars_milestone_cost' = '$37.5M'"
[TURN 4] THOUGHT: Ready to answer user.
         FINAL ANSWER: "Project Mars costs $37.5 million per milestone."
[TEST VERIFICATION] All 5 unit tests green.

Validate your work

Run the pytest test suite:

pytest tests/run_tests.sh

Troubleshooting

  1. Memory store persistence: Ensure memory_store is an instance dictionary that persists across all turns.
  2. Prompt rendering syntax: Verify that sliding window logic correctly compacts middle turns while retaining the system header.

Common mistakes

Practice assignment

Extend the agent with a Persistent Disk State Serializer that saves the complete trajectory and key-value memory store to a JSON file upon completion.

Extension challenge

Implement a Token Budget Guard that calculates approximate BPE token counts and dynamically triggers sliding window compaction whenever the prompt exceeds 2,000 estimated tokens.

Quiz

Q1. Why is building an agent from scratch without high-level frameworks valuable for production engineering?

  1. Full control of prompt, errors, budget, and state
  2. It makes the Python script run faster than C++
  3. Frameworks are forbidden in all enterprise companies
  4. It eliminates the need for unit tests
Show answer

Answer: A. Full control of prompt, errors, budget, and state

Scratch implementations demystify agent loops and provide maximum transparency and customization for production needs.

Q2. What is the purpose of sliding window memory compaction in an agent's working scratchpad?

  1. To delete all user messages
  2. Keep the system prompt and recent turns, prune the rest
  3. To encrypt the conversation history
  4. To increase the temperature parameter
Show answer

Answer: B. Keep the system prompt and recent turns, prune the rest

Sliding window memory prevents context overflow on extended multi-turn execution trajectories.

Q3. How does cycle detection identify that an agent is caught in a repetitive loop?

  1. By checking if CPU utilization reaches 100%
  2. By listening to audio signals
  3. Hash the thought-action tuple and look for a repeat
  4. By counting the number of vowels in the whole prompt
Show answer

Answer: C. Hash the thought-action tuple and look for a repeat

Hashing recent action states exposes repetitive non-progress loops so the runtime can abort or intervene.

Q4. What is the role of an explicit system prompt in a scratch agent?

  1. It restarts the server
  2. It compiles the Python interpreter
  3. It downloads packages from PyPI
  4. It defines the tools, the syntax, and the limits
Show answer

Answer: D. It defines the tools, the syntax, and the limits

The system prompt instructs the foundation model how to structure its reasoning and interact with tools.

Q5. What should the agent runtime do when maximum iterations are reached without a Final Answer?

  1. Exit cleanly with MAX_STEPS_EXCEEDED
  2. Hang indefinitely
  3. Crash the host machine
  4. Invent a hallucinated final answer
Show answer

Answer: A. Exit cleanly with MAX_STEPS_EXCEEDED

Graceful termination with a clear error status prevents infinite compute spend and aids debugging.

Glossary

Pure Python Agent
An agent runtime implemented using only the Python standard library with zero external dependencies.
Sliding Window Compactor
A memory strategy that maintains essential system prompts and recent turns while dropping old intermediate noise.
Regex Stream Extractor
Regular expressions that identify Thought, Action, and Final Answer sections in model responses.
Action Hash
A cryptographic or string hash of an action payload used for cycle detection.
Trajectory History
The complete chronological sequence of states, thoughts, actions, and observations in an execution run.
Deterministic Dispatcher
A dispatcher routing validated tool calls directly to their target Python callable.
Zero-Dependency
Software built entirely with core language primitives, ensuring maximum portability and minimal attack surface.
State Serialization
Converting an agent's memory and execution state into JSON for persistence or network transmission.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.