LLMs and Generative AI β€Ί LLM APIs β€Ί Day 256

Day 256: Tool Use and Function Calling

Day 256 of 365 β€” Tool Use and Function Calling

Master external tool execution and function calling: define JSON Schema tool definitions across Anthropic (tools) and OpenAI (tools) schemas, parse tool use stop reasons (tool_use / tool_calls), execute Python functions with typed JSON validation, return tool results back into the conversation context array (tool_result / role: "tool"), handle multi-tool parallel calls and recursive agent tool execution loops, and build an automated tool calling dispatcher in Python.

Course
LLMs and Generative AI
Category
LLM APIs
Reading time
β‰ˆ 35 min
Practical time
β‰ˆ 50 min
Lesson duration
1h 25m
Last verified
2026-08-30

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/llms-and-generative-ai/day-256-tool-use-and-function-calling

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path β€” section / subsection / week / day:
    cd labs/sections/llms-and-generative-ai/day-256-tool-use-and-function-calling
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work β€” read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

By themselves, language models are isolated neural networks frozen at the moment of their pretraining cutoff. A model cannot look up current stock prices, query your production SQL database, calculate a 50-digit trigonometric formula, or deploy a Kubernetes pod on its own.

Tool Use (Function Calling) bridges the gap between text generation and software execution. Instead of attempting to hallucinate answers, the model acts as an intelligent reasoning orchestrator:

Tool calling is the fundamental engine that powers autonomous AI agents, workflow automation, and enterprise assistants.

Today, you will master JSON Schema tool declarations across Anthropic and OpenAI standards, implement parallel tool dispatching, build recursive multi-step agent loops, and create an automated tool calling execution engine in Python.


The idea in plain language

Diagram: what tool access adds to a language model

Think of an architect designing a building:


Historical background

Tool calling execution loop architectural lifecycle diagram

  1. 2023 (ReAct & Raw Prompt Scraping): Early agent systems prompted models to output raw text strings like Action: Search[Tokyo Weather], relying on fragile regex string parsers that broke frequently.
  2. 2023 (OpenAI Function Calling): OpenAI introduced native function calling in gpt-4-0613, fine-tuning models to emit structured JSON argument payloads and introducing the finish_reason: "function_call" status.
  3. 2024 (Anthropic Tool Use & Parallel Tools): Anthropic released native Tool Use in Claude 3, standardizing multi-tool declarations and introducing parallel tool calling where models trigger multiple tools in a single turn.
  4. 2024 (Strict Schemas & Constrained Grammars): Providers added strict grammar masking (OpenAI Strict Tools / Outlines), guaranteeing that tool call parameters adhere 100% mathematically to the declared JSON schema.

What it is β€” and what it is not

What Tool Calling IS:

What it is NOT:


Why it was created and what problems it solves

Tool use solves 4 fundamental weaknesses of pure language models:

  1. Knowledge Cutoff Elimination: Access live real-time APIs (weather, stock tickers, news feeds).
  2. Deterministic Computation: Offload arithmetic, dates, and mathematical operations to Python calculators.
  3. Action Execution: Send emails, query SQL databases, update CRM records, or trigger cloud deployments.
  4. Hallucination Prevention: Ground answers in verified external truth retrieved during the conversation turn.

How it works

Let us dissect tool schema definitions, response parsing, and the multi-turn agent execution loop.


1. Tool Declaration Schemas (Anthropic vs. OpenAI)

Diagram: the four properties of a well-described tool definition

Anthropic Tool Definition Schema:

anthropic_tools = [
    {
        "name": "get_stock_price",
        "description": "Retrieves the real-time stock price and trading volume for a ticker symbol.",
        "input_schema": {
            "type": "object",
            "properties": {
                "ticker": {
                    "type": "string",
                    "description": "Stock ticker symbol, e.g. AAPL, GOOGL"
                },
                "currency": {
                    "type": "string",
                    "enum": ["USD", "EUR", "GBP"],
                    "default": "USD"
                }
            },
            "required": ["ticker"]
        }
    }
]

OpenAI Tool Definition Schema:

openai_tools = [
    {
        "type": "function",
        "function": {
            "name": "get_stock_price",
            "description": "Retrieves the real-time stock price for a ticker symbol.",
            "parameters": {
                "type": "object",
                "properties": {
                    "ticker": {"type": "string", "description": "Stock ticker, e.g. AAPL"}
                },
                "required": ["ticker"]
            }
        }
    }
]

2. The 4-Stage Tool Calling Lifecycle

Diagram: the four stages of a tool calling exchange

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                   TOOL CALLING 4-STAGE LIFECYCLE                       β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ STAGE 1: DECLARATION & INGRESS                                         β”‚
β”‚ - Client sends user prompt + list of available tool JSON schemas.      β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ STAGE 2: MODEL TOOL_USE EMISSION                                       β”‚
β”‚ - Model determines tool need; returns stop_reason="tool_use" with      β”‚
β”‚   tool ID, function name, and JSON argument dictionary.                β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ STAGE 3: LOCAL HOST EXECUTION & PACKAGING                              β”‚
β”‚ - Application dispatches local Python function with validated args;     β”‚
β”‚   formats result into tool_result message block.                       β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ STAGE 4: MODEL SYNTHESIS & FINAL ANSWER                                β”‚
β”‚ - Client appends tool_result to messages array and calls API again;   β”‚
β”‚   model returns final conversational answer (stop_reason="end_turn"). β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

3. Parallel Tool Calling Architecture

Diagram: how independent tool calls are executed in parallel

Parallel and recursive tool calling architecture diagram

When a user asks: β€œWhat is the weather in Tokyo, London, and New York?”


4. Recursive Multi-Step Agent Execution Loops

When solving complex multi-hop problems (e.g. β€œFind user 102’s recent order and check its tracking status”):

  1. Step 1: Model calls lookup_order(user_id=102) -> returns order_id="ORD-991".
  2. Step 2: Model receives ORD-991 and immediately calls get_tracking(order_id="ORD-991") -> returns "Out for delivery".
  3. Step 3: Model has all required data -> returns final answer to user.

Critical Guardrail: Always bound agent loops with max_iterations = 5 to prevent infinite loops and runaway billing costs!


5. Error Handling in Tool Results

When a local function fails (e.g. TimeoutError or DatabaseDown):


6. Pydantic-Backed Tool Dispatchers & Automatic Schema Generation

Manually handwriting raw JSON schema dictionaries in Python is error-prone. In modern production codebases, developers generate schemas automatically from Pydantic models:

from pydantic import BaseModel, Field
from typing import Literal

class StockPriceInput(BaseModel):
    ticker: str = Field(..., description="Stock ticker symbol, e.g. AAPL, NVDA")
    currency: Literal["USD", "EUR", "GBP"] = Field("USD", description="Settlement currency")

# Automatically generate standard JSON Schema
json_schema = StockPriceInput.model_json_schema()

7. Enforcing Specific Tool Choices (tool_choice)

You can control model tool invocation behavior precisely via the tool_choice parameter:


8. Security & Tool Sandboxing: Mitigating Prompt Injections in Tool Outputs

Diagram: the four controls that limit the damage from a compromised tool call

Connecting foundation models to execution environments creates critical security attack surfaces:


9. Dynamic Tool Pruning with Vector Embeddings

In enterprise environments with hundreds of internal microservices:


10. The Model Context Protocol (MCP) Standard

Diagram: bespoke tool integrations compared with a shared tool protocol

In late 2024, Anthropic open-sourced the Model Context Protocol (MCP):


An everyday analogy

Diagram: an assistant working from memory compared with one who can make calls

Think of a pilot flying an airplane:


Examples in practice

Let us build an Automated Tool Dispatcher & Agent Loop in Python:

from typing import Dict, Any, List, Callable
import json

class ToolDispatcher:
    def __init__(self):
        self.registry: Dict[str, Callable] = {}
        self.schemas: List[Dict[str, Any]] = []

    def register_tool(self, name: str, description: str, schema: Dict[str, Any], func: Callable):
        self.registry[name] = func
        self.schemas.append({
            "name": name,
            "description": description,
            "input_schema": schema
        })

    def execute_tool(self, name: str, args: Dict[str, Any]) -> str:
        if name not in self.registry:
            return f"Error: Tool '{name}' is not registered."
        try:
            return str(self.registry[name](**args))
        except Exception as e:
            return f"Error executing tool '{name}': {str(e)}"

    def run_agent_turn(self, user_prompt: str, mock_model_response: Dict[str, Any]) -> Dict[str, Any]:
        # Simulates one round of tool execution and result packaging
        tool_results = []
        if mock_model_response.get("stop_reason") == "tool_use":
            for call in mock_model_response.get("content", []):
                if call.get("type") == "tool_use":
                    out = self.execute_tool(call["name"], call["input"])
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": call["id"],
                        "content": out
                    })
        return {
            "tool_results": tool_results,
            "is_complete": len(tool_results) == 0
        }

Implications: security, privacy, performance, scalability, and cost

Diagram: the capability, security, latency and cost implications of tool calling

  1. Sandboxing & Least Privilege:
    • Execute tools in isolated environments with minimal necessary permissions. Never give a model unrestricted root shell access.
  2. Idempotency in Tools:
    • Design action tools to be idempotent so that network retries do not execute duplicate credit card charges or duplicate database records.

Alternatives: free, open source, and commercial

Framework / StandardProtocolParallel ExecutionBest For
Anthropic ToolsMessages APINative Multi-ToolStrict typing & Claude agents
OpenAI Function Calling/v1/chat/completionsNative Multi-ToolUniversal ecosystem tooling
LangChain ToolsPython DecoratorsAsync & ThreadedRapid prototyping
MCP (Model Context Protocol)JSON-RPC StandardDistributed ProcessUniversal IDE & agent tool sharing

Diagram: retrieval compared with tool calling as ways to extend a model

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                   TOOL INTEGRATION COMPARISON                          β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Feature            β”‚ Anthropic Tools β”‚ OpenAI Functions β”‚ ReAct Text   β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ Schema Standard    β”‚ JSON Schema D7  β”‚ JSON Schema D7   β”‚ Raw Strings  β”‚
β”‚ Strict Enforcement β”‚ Constrained FSM β”‚ Constrained FSM  β”‚ None (Regex) β”‚
β”‚ Parallel Tooling   β”‚ Yes             β”‚ Yes              β”‚ Fragile      β”‚
β”‚ Stop Reason Signal β”‚ "tool_use"      β”‚ "tool_calls"     β”‚ Text parsing β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

When to use it β€” and when not to

Diagram: when tool access is warranted

When to Use Function Calling:

When NOT to use:


The AI connection

Knowledge check

  1. Does the LLM execute Python functions directly on its GPU servers?
  2. What is the purpose of the unique tool_use_id returned in a tool call?
  3. How does Parallel Tool Calling improve overall application latency?
  4. Why must recursive agent execution loops enforce a maximum iteration limit?
  5. How should application runtime exceptions be communicated back to the model?

Hands-on exercise

In this lab, you will build and test an Automated Tool Dispatcher and Agent Loop in Python: register custom mathematical and lookup tools with JSON Schema declarations, parse simulated tool invocation payloads, execute Python handlers with parameter validation, package structured tool_result messages, and verify output assertions against automated test suites.

Expected output

[Tool Use and Function Calling Verification Suite]
Testing Tool Registration & Schema Validation:
  Registered Tool: 'calculate_tax' (Params: amount, state) [VALIDATED]
  Registered Tool: 'get_order_status' (Params: order_id) [VALIDATED]
Testing Tool Dispatch Execution:
  Input: {'amount': 100.0, 'state': 'CA'} -> Output: '107.25' [PASS]
Testing Error Handling on Invalid Tool:
  Input: 'unknown_tool' -> Caught error message [PASS]
Testing Agent Loop Result Packaging:
  tool_result ID: 'toolu_tax_01' | Result: '107.25' [VERIFIED]
Test Suite: 3 passed in 0.38s

Validate your work

Run the automated test runner:

./tests/run_tests.sh

Troubleshooting

Common mistakes


Practice assignment

  1. Implement an Async Parallel Tool Dispatcher using asyncio.gather that executes 5 independent tool calls simultaneously in sub-second time.
  2. Build an automated SQL Query Sanitizer Tool that validates SQL queries against a strict read-only AST parser before executing against SQLite.

Extension challenge

Implement a Human-in-the-Loop Multi-Tool Approval Workflow:

Quiz

Q1. Does a foundation model (e.g. Claude or GPT-4o) execute external Python code directly on the model hosting server?

  1. Yes: via a hidden SSH terminal connection
  2. Yes: the model runs arbitrary Python code on GPU memory
  3. No β€” it emits a call; your code runs the function
  4. No: tools can only execute JavaScript
Show answer

Answer: C. No β€” it emits a call; your code runs the function

Models only decide which tool to call and formulate the arguments; local application code executes the tool.

Q2. What is the role of tool_call_id (or id) in the tool calling protocol?

  1. It stores the user password
  2. It measures token latency
  3. It encrypts the model weights
  4. It correlates the call with its tool_result
Show answer

Answer: D. It correlates the call with its tool_result

The tool call ID ensures the model maps the returned output to the exact tool call requested.

Q3. What is Parallel Tool Calling?

  1. Emitting several independent tool calls in one turn
  2. Running two LLMs at the same time
  3. Using two computers to type prompts
  4. Executing tools in binary code
Show answer

Answer: A. Emitting several independent tool calls in one turn

Parallel tool calling allows models to batch independent operations in a single inference step.

Q4. Why is it critical to enforce a max_iterations boundary on recursive agent tool execution loops?

  1. Because Python loops crash after 10 turns
  2. To stop infinite tool loops burning budget
  3. Because API keys expire after 5 minutes
  4. To clear the GPU cache
Show answer

Answer: B. To stop infinite tool loops burning budget

Bounding loop iterations protects against runaway recursion and infinite billing spikes.

Q5. How should exceptions during tool execution (e.g. DatabaseConnectionError) be communicated back to the model?

  1. Delete the conversation history
  2. Crash the client Python script immediately
  3. Return it as a tool_result with is_error set
  4. Send a blank string
Show answer

Answer: C. Return it as a tool_result with is_error set

Returning structured error messages in tool results enables the model to self-correct or notify the user gracefully.

Glossary

Tool Use / Function Calling
The capability of an LLM to detect when an external utility is needed, select the tool, and emit structured execution arguments.
JSON Schema
A standard declarative vocabulary for defining and validating the data types, required fields, and constraints of tool parameters.
tool_use / tool_calls
The structured content block emitted by the model specifying the tool name, unique ID, and argument dictionary.
tool_result / role: tool
The message block returned by the host application containing the output payload of the executed function.
Tool Dispatcher
The client-side registry mapping tool string names to executable Python functions with parameter validation.
Parallel Tool Calling
Emitting multiple tool invocations simultaneously within a single response turn to optimize latency.
Agent Execution Loop
A cyclical multi-turn loop that repeatedly executes tools and feeds results back to the model until a final answer is synthesized.
Strict Tool Schema
Enforcing 100% adherence to JSON schemas during generation via constrained grammar decoding.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.