AI Engineering: Agents and Applications › AI Agents › Day 290

Day 290: Designing Tools for Agents

Day 290 of 365 — Designing Tools for Agents

Master production tool design for AI agents: craft robust JSON Schemas, implement automated parameter coercion and validation, build idempotency caching, and isolate dangerous execution side effects.

Course
AI Engineering: Agents and Applications
Category
AI Agents
Reading time
≈ 35 min
Practical time
≈ 50 min
Lesson duration
1h 25m
Last verified
2026-08-30

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/ai-engineering/day-290-designing-tools-for-agents

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/ai-engineering/day-290-designing-tools-for-agents
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

In an autonomous agent architecture, the foundation model acts as the cognitive cortex, but external tools are its physical hands, eyes, and sensory organs. An agent without tools is locked inside its static training weights, unable to query fresh databases, inspect live production servers, or write code to the local filesystem. However, building reliable tools for non-deterministic AI agents is vastly different from building traditional REST APIs or microservice libraries for human developers.

When human engineers consume an API, they read comprehensive documentation portals, inspect interactive Swagger UIs, and debug type errors with static compilers. An LLM agent, by contrast, perceives a tool exclusively through a compact JSON Schema embedded within its dynamic system prompt. If parameter names are vague, if descriptions lack semantic clarity, or if error responses fail to explain formatting discrepancies, the model will hallucinate invalid arguments, enter repetitive failure loops, and corrupt downstream application state.

Mastering production tool design—defining strict JSON Schemas, implementing parameter coercion, separating read-only queries from side-effecting mutations, enforcing idempotency keys, and establishing execution sandboxes—is the hallmark of senior AI systems engineering.

Agent Tool Schema and Validation illustrating JSON Schema definition, LLM tool selection, parameter coercion, and sandbox execution

The idea in plain language

Diagram: a vague tool definition compared with a precise one

Diagram: the four properties of a production-grade agent tool

Imagine handing a set of specialized tools to an apprentice carpenter:

In agentic systems, well-designed tools provide clear descriptions, strict type validation, safe default parameters, and actionable error messages that guide the agent back to success.

Historical background

Diagram: the progression from regex-scraped actions to the Model Context Protocol

The evolution of agent-tool interfaces has evolved through three distinct phases:

1. The Raw Text & Regex Scraping Era (2022)

In early ReAct experiments, tools were defined as unstructured text snippets in the system prompt. Models were instructed to emit arbitrary strings like Action: Search[Tokyo population]. The runtime relied on fragile regular expressions to extract arguments. Unescaped brackets, commas in queries, or multi-line strings frequently broke parsing pipelines.

2. OpenAI Function Calling & Native JSON Schema (June 2023)

In June 2023, OpenAI introduced fine-tuned Function Calling support in GPT-4 and GPT-3.5. Instead of relying on prompt hacking, models were trained specifically to output structured JSON adhering to standard JSON Schema specifications. This was rapidly adopted across the industry by Anthropic (Tool Use), Google Gemini, and open-source models (Hermes, Mistral, Qwen).

3. The Model Context Protocol (MCP) Standard (2024–Present)

In late 2024, Anthropic open-sourced the Model Context Protocol (MCP), standardizing how tools, database resources, and system prompts are exposed to LLM clients over JSON-RPC 2.0. MCP eliminated proprietary SDK lock-in, enabling a single tool server (such as a GitHub or PostgreSQL connector) to serve Claude, Gemini, and local coding agents interchangeably.

What it is — and what it is not

Let us establish strict technical criteria for agent tool architecture:

What an Agent Tool Is:

What an Agent Tool Is Not:

Diagram: a curated tool registry compared with an exposed API surface

+-------------------------------------------------------------------------------+
|                       TOOL DESIGN CRITERIA MATRIX                             |
+---------------------+-----------------------+---------------------------------+
| Dimension           | Anti-Pattern          | Production Best Practice        |
+---------------------+-----------------------+---------------------------------+
| Description Length  | "Searches db"         | Comprehensive semantic purpose  |
| Parameter Types     | Any / Object          | Strict Enums, Ints, ISO Dates   |
| Error Handling      | Returns 500 / Stacktrace| Returns Actionable Guidance   |
| Side Effects        | Unsafe Mutations      | Idempotency Key Required        |
| Return Size         | 50MB raw dump         | Compact Summaries / Paginated   |
+---------------------+-----------------------+---------------------------------+

Why it was created and what problems it solves

Production tool engineering resolves four systemic reliability hazards:

1. Tool Selection Confusion (Semantic Overlap)

When an agent is equipped with multiple tools featuring overlapping descriptions (e.g. search_database, query_sql, and find_records), the model suffers from cognitive interference, selecting suboptimal tools or hallucinating hybrid signatures. Explicit, mutually exclusive tool descriptions eliminate ambiguity.

2. Parameter Hallucination and Format Mismatches

Language models frequently emit dates as 08/30/2026 when an API expects 2026-08-30 (ISO-8601), or pass strings "50" where integers 50 are required. Production tool wrappers perform automated parameter coercion and schema pre-validation.

3. Accidental State Mutation (The Double-Charge Problem)

If an agent calls charge_credit_card(amount=50) and the network connection drops before receiving the response, the agent may retry the call three times, billing the customer $150. Production side-effecting tools enforce Idempotency Keys, ensuring duplicate requests return cached results without re-executing transactions.

4. Context Window Bloat

If a search tool returns a 50,000-line JSON payload, it saturates the model’s context window, degrading attention performance and multiplying token billing. Production tools implement automatic truncation, schema projection, and pagination.

How it works

Let us inspect the JSON Schema structure, parameter validation pipeline, and idempotency key caching mechanics:

[System Prompt Tool Registration: JSON Schema]
{
  "name": "fetch_user_profile",
  "description": "Retrieves verified user profile data by ID or email.",
  "parameters": {
    "type": "object",
    "properties": {
      "user_id": {"type": "string", "description": "Unique UUID of user"},
      "include_billing": {"type": "boolean", "default": false}
    },
    "required": ["user_id"]
  }
}
                        │
                        ▼
      [Model Generates Structured Tool Call]
      {"name": "fetch_user_profile", "arguments": "{\"user_id\": \"u-123\"}"}
                        │
                        ▼
      [Parameter Validator & Type Coercer]
      - Validates required properties present
      - Coerces string booleans ("true" -> True)
      - Enforces boundary limits
                        │
                        ▼
       [Idempotency & Execution Guard]
      - Read-only? Direct Execution
      - Mutating? Check Idempotency Key Cache
                        │
                        ▼
         [Sandbox Execution & Return]
      - Execute Python / SQL with 2000ms timeout
      - Return compact JSON observation to Agent

1. Writing High-Accuracy JSON Schemas

A production JSON Schema must specify:

  1. Clear high-level purpose and expected operational preconditions.
  2. Required vs optional properties with deterministic default values.
  3. Explicit constraints (enum, minimum, maximum, pattern).

Diagram: the three components of a high-accuracy tool schema

{
  "name": "create_calendar_event",
  "description": "Schedules a new meeting in the user's primary calendar. Returns the created event ID and conference URL.",
  "parameters": {
    "type": "object",
    "properties": {
      "title": {
        "type": "string",
        "description": "The concise meeting title or subject line."
      },
      "start_time": {
        "type": "string",
        "description": "Start timestamp in ISO-8601 format (e.g. '2026-08-30T14:00:00Z')."
      },
      "duration_minutes": {
        "type": "integer",
        "description": "Duration in minutes (must be between 5 and 480).",
        "minimum": 5,
        "maximum": 480
      },
      "attendees": {
        "type": "array",
        "items": {"type": "string"},
        "description": "List of attendee email addresses."
      }
    },
    "required": ["title", "start_time", "duration_minutes"]
  }
}

Idempotent Tool Execution Pipeline showing idempotency key caching, duplicate request suppression, and state mutation safety

2. Idempotency Key Enforcement

For mutating actions (POST, PUT, DELETE), every tool call signature accepts an optional idempotency_key: str. If omitted by the model, the runtime computes a deterministic hash of the tool name and sorted arguments: \text(Key) = \text(SHA256)(\text(tool_name) + \text(canonical_json)(\text(args))) If the key exists in the short-term cache, the cached response is returned immediately without re-executing the side effect.

Diagram: how an idempotency key prevents a retried tool call from repeating a side effect

An everyday analogy

Diagram: an unstructured order field compared with a structured ordering interface

Think of designing agent tools like creating an electronic ordering kiosk at a fast-food restaurant:

Structured schemas remove ambiguity and guarantee the kitchen (the backend) receives valid, executable orders every time.

Examples in practice

Let us inspect a Python implementation of a Tool Registry with JSON Schema validation, parameter coercion, and idempotency key caching:

import hashlib
import json
import time
from typing import Dict, Any, Callable, Optional, List

class ToolDefinition:
    def __init__(self, name: str, description: str, schema: Dict[str, Any], func: Callable, is_mutating: bool = False):
        self.name = name
        self.description = description
        self.schema = schema
        self.func = func
        self.is_mutating = is_mutating

class ProductionToolRegistry:
    def __init__(self, cache_ttl_seconds: int = 300):
        self.tools: Dict[str, ToolDefinition] = {}
        self.idempotency_cache: Dict[str, Dict[str, Any]] = {}
        self.cache_ttl = cache_ttl_seconds
        
    def register(self, tool: ToolDefinition):
        self.tools[tool.name] = tool
        
    def get_schemas(self) -> List[Dict[str, Any]]:
        return [
            {
                "name": t.name,
                "description": t.description,
                "parameters": t.schema
            }
            for t in self.tools.values()
        ]
        
    def validate_and_coerce(self, tool: ToolDefinition, args: Dict[str, Any]) -> Tuple[bool, Dict[str, Any], str]:
        schema = tool.schema
        properties = schema.get("properties", {})
        required = schema.get("required", [])
        
        # Check required fields
        for req in required:
            if req not in args:
                return False, {}, f"Missing required parameter: '{req}'"
                
        coerced = {}
        for k, v in args.items():
            if k not in properties:
                continue # Ignore undefined params or return error
            target_type = properties[k].get("type")
            try:
                if target_type == "integer":
                    coerced[k] = int(v)
                elif target_type == "number":
                    coerced[k] = float(v)
                elif target_type == "boolean":
                    if isinstance(v, str):
                        coerced[k] = v.lower() in ("true", "1", "yes")
                    else:
                        coerced[k] = bool(v)
                elif target_type == "string":
                    coerced[k] = str(v)
                else:
                    coerced[k] = v
            except (ValueError, TypeError) as err:
                return False, {}, f"Type coercion failed for parameter '{k}': expected {target_type}, got {type(v).__name__}"
                
        return True, coerced, "OK"
        
    def execute(self, tool_name: str, raw_args: Dict[str, Any], idempotency_key: Optional[str] = None) -> Dict[str, Any]:
        if tool_name not in self.tools:
            return {"success": False, "error": f"Tool '{tool_name}' not registered."}
            
        tool = self.tools[tool_name]
        valid, clean_args, err_msg = self.validate_and_coerce(tool, raw_args)
        if not valid:
            return {"success": False, "error": err_msg}
            
        # Idempotency Check for Mutating Actions
        if tool.is_mutating:
            if not idempotency_key:
                # Generate key from canonical hash
                canon_str = f"{tool_name}:{json.dumps(clean_args, sort_keys=True)}"
                idempotency_key = hashlib.sha256(canon_str.encode()).hexdigest()
                
            now = time.time()
            if idempotency_key in self.idempotency_cache:
                cached = self.idempotency_cache[idempotency_key]
                if now - cached["timestamp"] < self.cache_ttl:
                    return {"success": True, "result": cached["result"], "cached": True}
                    
        # Execute in sandbox
        try:
            res = tool.func(**clean_args)
            if tool.is_mutating and idempotency_key:
                self.idempotency_cache[idempotency_key] = {"result": res, "timestamp": time.time()}
            return {"success": True, "result": res, "cached": False}
        except Exception as e:
            return {"success": False, "error": f"Execution error in {tool_name}: {str(e)}"}

Implications: security, privacy, performance, scalability, and cost

Diagram: the privilege, output and selection hazards of agent tools

1. Tool-Level Privilege Escalation

Never give an agent a tool named run_raw_sql that accepts arbitrary SQL strings with write access to production databases. Instead, expose parameterized domain-specific tools like get_order_by_id(order_id: str) or update_customer_address(...) with strict permission validation.

2. Output Payload Truncation

If a document retrieval tool returns a 10MB PDF extract, the agent’s context window will instantly overflow. Production tool runners enforce a maximum output character ceiling (e.g. max 4,000 characters), summarizing or chunking larger returns.

Tool Category            Permitted Methods    Timeout Ceiling    Max Output Characters
---------------------------------------------------------------------------------------
Read-Only Query          SELECT / GET         1,500 ms           8,000 chars
File System Reader       cat / read_file      500 ms             4,000 chars
Side-Effecting Action    INSERT / POST        3,000 ms           1,000 chars (Status + ID)
Shell Execution Sandbox  Isolated CLI         5,000 ms           2,500 chars (Tail log)

Alternatives: free, open source, and commercial

Diagram: provider-native function calling compared with the Model Context Protocol

Standard / Tool SystemProtocolStrengthsTradeoffs
OpenAI Function CallingNative JSON SchemaIndustry standard, wide model supportVendor-specific payload envelopes
Model Context Protocol (MCP)JSON-RPC 2.0Universal cross-client portabilityRequires running MCP server daemon
LangChain ToolsPython Pydantic BaseToolRich ecosystem of community connectorsHeavy dependency footprint
Custom Schema RegistryPure Python JSON SchemaZero dependencies, instant auditingRequires custom client serialization
+-----------------------+-----------------------+-----------------------+
| Feature               | Traditional REST API  | Agent Tool            |
+-----------------------+-----------------------+-----------------------+
| Primary Consumer      | Human Developer       | Stochastic Language Model|
| Documentation         | Markdown / Swagger UI | In-Prompt JSON Schema |
| Parameter Tolerance   | Strict (Throws 400)   | Coercive + Explanatory|
| Response Format       | Large JSON Payloads   | Compact Semantic Text |
| Mutation Safety       | Manual Idempotency    | Automated Idempotency |
+-----------------------+-----------------------+-----------------------+

When to use it — and when not to

Diagram: when a full tool registry with validation and idempotency is warranted

Choose Production Tool Registries when:

Do NOT use Complex Tool Registries when:

The AI connection

Knowledge check

  1. Why is parameter coercion essential in agent tool registries? Because language models frequently emit numeric or boolean arguments as strings; coercion normalizes types without rejecting the call.
  2. What does an idempotency key prevent during network retries? It prevents duplicate execution of side-effecting operations (such as multiple credit card charges or duplicate database records).
  3. What are the key elements of a production JSON Schema for an agent tool? Name, clear semantic description, typed property definitions with constraints, and an explicit list of required fields.

Hands-on exercise

In this lab, you will implement a Production Tool Registry with JSON Schema Validation, Parameter Coercion, and Idempotency Key Caching in Python. Your implementation will:

  1. Define a ToolDefinition dataclass supporting both read-only and mutating actions.
  2. Implement schema validation and type coercion (converting strings to integers, booleans, and floats).
  3. Build an Idempotency Cache that suppresses duplicate side-effect calls within a configurable TTL window.
  4. Verify execution safety and error handling across a suite of unit tests.

Expected output

============================= test session starts ==============================
collected 5 items

tests/test_tool_registry.py .....                                        [100%]

============================== 5 passed in 0.05s ===============================
[TOOL REGISTRY] Registered 3 tools: ['calculate', 'query_db', 'create_order']
[VALIDATION] Parameter 'count' successfully coerced from "5" to int(5).
[IDEMPOTENCY] First call to create_order(order_id="101") executed successfully.
[IDEMPOTENCY] Duplicate call to create_order(order_id="101") suppressed -> CACHED RESULT RETURNED.
[ERROR HANDLING] Missing required parameter 'sql' returned actionable schema error.

Validate your work

Run the pytest test suite:

pytest tests/run_tests.sh

Troubleshooting

  1. Type coercion error on boolean values: Handle string literals like "true", "false", "1", and "0" explicitly.
  2. Idempotency cache collision: Ensure canonical JSON serialization uses sort_keys=True.

Common mistakes

Practice assignment

Extend the Tool Registry to support Rate Limiting per tool (e.g. max 5 calls per minute for external API scrapers).

Extension challenge

Implement a Schema Generator that uses Python type annotations and docstrings (inspect.signature) to automatically generate valid JSON Schema definitions from vanilla Python functions.

Quiz

Q1. Why is JSON Schema considered the industry standard for declaring agent tools?

  1. Because it is written in binary
  2. A machine-readable spec of names, types, and fields
  3. It compiles the Python source into Rust automatically
  4. It bypasses all cloud security firewalls
Show answer

Answer: B. A machine-readable spec of names, types, and fields

JSON Schema provides a formal standard that foundation models use to construct typed function arguments.

Q2. What is tool idempotency and why is it vital for autonomous agent stability?

  1. Idempotency deletes old logs
  2. Idempotency makes tools run at 10 GHz
  3. Repeating a call causes no duplicate side effects
  4. Idempotency compresses JSON text
Show answer

Answer: C. Repeating a call causes no duplicate side effects

Idempotency prevents dangerous duplicate operations (e.g. charging a credit card twice) if an agent retries an action.

Q3. How should parameter descriptions in a tool JSON Schema be written for optimal model performance?

  1. Left blank to save tokens
  2. Containing obfuscated code
  3. Written in hexadecimal
  4. Clear, with units, ranges, and examples
Show answer

Answer: D. Clear, with units, ranges, and examples

Detailed descriptions with units and format constraints dramatically reduce argument hallucination.

Q4. What is the purpose of automated parameter coercion in a tool registry?

  1. To cast strings to the expected types
  2. To delete invalid parameters
  3. To restart the whole operating system first
  4. To translate English to Spanish
Show answer

Answer: A. To cast strings to the expected types

Models occasionally emit numbers as strings; automatic coercion prevents unnecessary type error failures.

Q5. Which tool design principle prevents destructive side effects in production environments?

  1. Granting root administrative access to all tools
  2. Read-only defaults, sandboxing, confirmation gates
  3. Disabling all unit tests
  4. Increasing the batch size
Show answer

Answer: B. Read-only defaults, sandboxing, confirmation gates

Restricting tool privileges and requiring confirmation for destructive mutations protects system integrity.

Glossary

JSON Schema
A standard declarative format describing the structure, types, and constraints of tool arguments.
Tool Registry
A centralized management system registering, validating, and dispatching tool invocations.
Idempotency Key
A unique identifier (such as a parameter hash) ensuring repeated executions do not cause duplicate mutations.
Parameter Coercion
Automatically casting incoming arguments to their schema-defined types safely.
Sandbox Isolation
Executing untrusted or high-risk tool operations within isolated containers or restricted environments.
Destructive Action Gate
A security boundary requiring human review before executing irreversible operations.
Function Calling API
A model inference capability structured specifically to output valid tool invocations.
Error Contract
Standardized JSON error responses formatted specifically to guide model self-correction.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.