AI Engineering: Agents and Applications › AI Agents › Day 289

Day 289: The Agent Loop: Reason, Act, Observe

Day 289 of 365 — The Agent Loop: Reason, Act, Observe

Master the inner mechanics of the ReAct (Reason + Act) loop: build custom text and JSON stream parsers, implement self-healing error observation feedback, and enforce termination predicates.

Course
AI Engineering: Agents and Applications
Category
AI Agents
Reading time
≈ 35 min
Practical time
≈ 50 min
Lesson duration
1h 25m
Last verified
2026-08-30

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/ai-engineering/day-289-the-agent-loop-reason-act-observe

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/ai-engineering/day-289-the-agent-loop-reason-act-observe
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

In classical computer science, algorithms execute deterministic sequences of instructions compiled into machine bytecode. In modern AI engineering, an agent loop is a dynamic cybernetic feedback loop where the language model acts as a real-time stochastic controller. Rather than following a rigid script, the model must continuously inspect the environment, formulate deductive hypotheses, execute external tools, interpret unstructured observation signals, and self-correct when unexpected errors occur.

The foundational paradigm underpinning modern agents is ReAct (Reasoning and Acting). Introduced by Yao et al. in 2022, ReAct established that decoupling an agent’s internal deduction (Thought:) from its environmental actuation (Action:) dramatically enhances accuracy, mitigates hallucination, and enables robust multi-step problem solving. However, moving from an academic paper to an enterprise production agent requires engineering rigor: building high-throughput token stream parsers, handling malformed tool schemas, implementing exponential backoff retry handlers, and preventing runaway execution loops.

Understanding the ReAct loop mechanics allows AI engineers to construct resilient autonomous assistants that can reliably execute multi-file software refactorings, deep financial research investigations, and automated infrastructure incident triage.

ReAct Agent Execution Loop showing Thought, Action, Environment Execution, Observation, and Dynamic Termination Loop

The idea in plain language

Diagram: pure reasoning compared with blind action, and why interleaving beats both

Diagram: the alternating structure of a ReAct execution loop

Imagine you are a detective investigating a complex puzzle in an escape room:

The ReAct loop is this rigorous, alternating dance of thinking before doing, and reflecting after seeing the results.

Historical background

Diagram: the progression from chain-of-thought to Plan-and-Solve and Reflexion

The evolution of agent execution control loops has traversed three distinct architectural generations:

1. Chain-of-Thought and Pure Reasoning (2022)

In January 2022, Wei et al. introduced Chain-of-Thought (CoT) Prompting, demonstrating that prompting models to “think step-by-step” unlocked emergent multi-step arithmetic and symbolic logic. However, CoT was completely isolated from external reality—the model could not verify its reasoning against external tools or fresh web data.

2. The ReAct Breakthrough (October 2022)

Shunyu Yao and colleagues at Princeton and Google Brain published ReAct: Synergizing Reasoning and Acting in Language Models. ReAct proved that interleaving explicit reasoning traces with action calls created a synergistic effect: reasoning traces helped the model track sub-goals and select tools, while external observations grounded the reasoning and prevented hallucination drift.

3. Plan-and-Solve and Reflexion (2023–2024)

As tasks grew more intricate, two key extensions emerged:

What it is — and what it is not

Let us formalize the boundaries of agent execution loops:

What the ReAct Loop Is:

What the ReAct Loop Is Not:

Diagram: a ReAct loop compared with three simpler structures mistaken for it

+-------------------------------------------------------------------------------+
|                       AGENT CONTROL LOOP TAXONOMY                             |
+---------------------+-----------------------+---------------------------------+
| Strategy            | Planning Horizon      | Error Recovery Mechanism        |
+---------------------+-----------------------+---------------------------------+
| Single-Step Prompt  | Immediate Next Token  | None                            |
| Chain-of-Thought    | Unrolled in Prompt    | Purely Internal Deduction       |
| Pure ReAct          | 1-Step Lookahead      | Immediate Observation Feedback  |
| Plan-and-Solve      | Global Task DAG       | Dynamic Re-Planning on Failure  |
| Reflexion           | Multi-Episode Memory  | Verbal Post-Mortem Reflection   |
+---------------------+-----------------------+---------------------------------+

Why it was created and what problems it solves

Diagram: the four failure modes of single-pass generation that the ReAct loop addresses

The ReAct loop resolves four universal failure modes in autonomous software execution:

1. The Greedy Myopia Problem

When an LLM attempts to solve a 10-step problem in a single generation, errors in step 2 silently cascade and corrupt all subsequent steps. ReAct isolates each step into a distinct turn: the model executes step 1, verifies the real-world output, and only then formulates step 2 conditioned on ground truth.

2. Hallucinated Tool Signatures

Without an explicit reasoning buffer, language models frequently hallucinate non-existent API parameters or invent fictional tool names. Giving the model space to think ("I need to call search_users with email parameter") dramatically increases schema compliance.

3. Syntax Error Recovery (Self-Healing JSON)

If an agent generates malformed JSON ({"query": "data",} with a trailing comma), a naive runtime crashes. A ReAct engine catches the json.JSONDecodeError, formats an observation: "Observation: Error - invalid JSON: trailing comma at line 1 column 18", allowing the model to correct its syntax on the next step.

4. Convergence and Goal Completion Verification

A ReAct agent explicitly evaluates when the objective has been fully satisfied. Rather than guessing when to stop, the agent inspects the accumulated observations and determines whether sufficient evidence exists to emit the Final Answer:.

How it works

Let us dissect the mechanics of prompt construction, token stream parsing, and action execution:

[System Prompt: Available Tools + ReAct Instructions]
[User Goal: "Find the population of Tokyo and calculate 15% of it"]
                        │
                        ▼
      [LLM Inference Call (Temperature = 0.0)]
                        │
                        ▼
  [Generated Output Stream: Thought + Action]
  "Thought: I need to search for Tokyo's population.
   Action: search_kb({"query": "Tokyo population 2024"})"
                        │
                        ▼
            [Deterministic Regex / JSON Parser]
                        │
         ┌──────────────┴──────────────┐
         ▼ (Valid Action)              ▼ (Syntax Error)
[Tool Sandbox Dispatch]         [Inject Syntax Error Obs]
         │                             │
         ▼                             │
[Observation Returned]                 │
"Observation: Tokyo pop: 14M"          │
         │                             │
         └──────────────┬──────────────┘
                        ▼
      [Append Step to Trajectory Scratchpad]
                        │
                        ▼
      [Stopping Condition Check: Step <= 10]
                        │
                        ▼
             (Loop to LLM Inference)

1. The ReAct Prompt Format Contract

A standard zero-dependency ReAct prompt establishes a strict grammar:

Answer the user's request using the following tools:
- {tool_name}: {tool_description} - Parameters: {json_schema}

Use the following strict format:
Thought: Describe your reasoning about what to do next.
Action: tool_name({"param": "value"})
Observation: The result will be provided by the environment.
... (this Thought/Action/Observation can repeat N times)
Thought: I have gathered all necessary information to answer.
Final Answer: The final response to the user.

2. Action Parsing & Grammar Extraction

In production, extracting tool calls from raw model text requires robust regex pattern matching. A robust parser handles both single-line JSON and multi-line fenced code blocks:

import re, json

![Diagram: what a production action parser must handle](./assets/parsing-contract.svg)

ACTION_REGEX = re.compile(r"Action:\s*([a-zA-Z0-9_]+)\((.*)\)", re.DOTALL)

def parse_action_string(text: str):
    match = ACTION_REGEX.search(text)
    if not match:
        return None
    tool_name = match.group(1).strip()
    raw_args = match.group(2).strip()
    try:
        args = json.loads(raw_args)
        return tool_name, args
    except json.JSONDecodeError as e:
        return tool_name, f"JSON_PARSE_ERROR: {str(e)}"

Plan and Solve vs ReAct Comparison diagram contrasting upfront macro planning with iterative step-by-step reasoning

3. Self-Correction via Observation Feedback

When the parser encounters a JSON_PARSE_ERROR, the runtime does not crash. It immediately appends an observation to the prompt: Observation: Tool execution failed. JSON syntax error: Expecting property name enclosed in double quotes. Please re-emit Action with valid JSON. On the next cycle, the model adjusts its formatting and emits valid syntax.

An everyday analogy

Diagram: a cook checking intermediate results compared with blind execution

Think of the ReAct loop like a chef cooking a complex gourmet dish:

If the chef tried to pour all ingredients into a cold pot at once without checking intermediate temperatures (blind execution), the sauce would split and fail.

Examples in practice

Let us inspect a complete, zero-dependency ReAct Execution Engine in Python:

import re
import json
from typing import Dict, Any, Tuple, Optional

class ReActEngine:
    def __init__(self, tools: Dict[str, Any], max_iterations: int = 6):
        self.tools = tools
        self.max_iterations = max_iterations
        self.trajectory = []
        
    def parse_generation(self, text: str) -> Tuple[str, Optional[str], Optional[Dict[str, Any]], Optional[str]]:
        # Extract Thought
        thought_match = re.search(r"Thought:\s*(.*?)(?=
Action:|
Final Answer:|$)", text, re.DOTALL)
        thought = thought_match.group(1).strip() if thought_match else ""
        
        # Check for Final Answer
        if "Final Answer:" in text:
            final_match = re.search(r"Final Answer:\s*(.*)", text, re.DOTALL)
            final_answer = final_match.group(1).strip() if final_match else text
            return "FINAL", thought, None, final_answer
            
        # Extract Action
        action_match = re.search(r"Action:\s*([a-zA-Z0-9_]+)\s*\((.*)\)", text, re.DOTALL)
        if action_match:
            tool_name = action_match.group(1).strip()
            args_raw = action_match.group(2).strip()
            try:
                args = json.loads(args_raw) if args_raw else {}
                return "ACTION", thought, {"tool": tool_name, "args": args}, None
            except json.JSONDecodeError as err:
                return "SYNTAX_ERROR", thought, {"tool": tool_name, "raw": args_raw, "error": str(err)}, None
                
        return "UNKNOWN", thought, None, None
        
    def execute_step(self, mock_llm_response: str) -> Dict[str, Any]:
        step_type, thought, action_data, final_ans = self.parse_generation(mock_llm_response)
        
        if step_type == "FINAL":
            return {"status": "COMPLETE", "final_answer": final_ans, "thought": thought}
        elif step_type == "ACTION":
            tool = action_data["tool"]
            args = action_data["args"]
            if tool in self.tools:
                try:
                    res = self.tools[tool](**args)
                    obs = f"Observation: {res}"
                except Exception as e:
                    obs = f"Observation: Error executing tool {tool}: {str(e)}"
            else:
                obs = f"Observation: Tool '{tool}' does not exist in registry."
            return {"status": "CONTINUE", "thought": thought, "tool": tool, "observation": obs}
        elif step_type == "SYNTAX_ERROR":
            obs = f"Observation: JSON parse error in action parameters: {action_data['error']}. Please retry with valid JSON."
            return {"status": "CONTINUE", "thought": thought, "observation": obs}
        else:
            obs = "Observation: Format error. Expected 'Action: tool_name(args)' or 'Final Answer: answer'."
            return {"status": "CONTINUE", "thought": thought, "observation": obs}

Implications: security, privacy, performance, scalability, and cost

1. Infinite Token Expansion in ReAct Loops

Every iteration in a ReAct loop appends (Thought + Action + Observation) to the prompt context. By step 8, the prompt context contains the entire historical transcript, scaling input token processing quadratically: \text(Total Tokens) = sum_(k=1)^K (\text(Base Prompt) + k cdot \text(Step Tokens)) To optimize cost and latency, production engines implement Scratchpad Truncation, summarizing older observations while preserving atomic fact extractions.

Diagram: how a ReAct scratchpad accumulates across steps

2. Tool Execution Sandboxing

Because actions execute real Python code, shell commands, or database queries, tool executors must be isolated within restricted Docker containers or WebAssembly sandboxes with strict execution timeouts (e.g. max 3.0 seconds per tool call).

Diagram: sandboxed tool execution compared with direct execution

Component                Latency Budget    Timeout Limit    Security Sandbox
-----------------------------------------------------------------------------
Local Python Math Tool   < 5 ms            50 ms            In-process eval (__builtins__: None)
Database Query Tool      < 50 ms           1000 ms          Read-only connection pool
Web Search / Scraper     < 400 ms          3000 ms          Isolated HTTP worker with domain allowlist
Shell Command Exec       < 200 ms          5000 ms          Rootless Docker container

Alternatives: free, open source, and commercial

Diagram: prompt-based ReAct compared with native tool-calling APIs

ApproachLoop PatternOverheadBest Suited For
Vanilla ReActIterative 1-StepMinimalGeneral problem solving, search agents
Plan-and-SolveMacro DAG + Sub-tasksModerateComplex multi-stage workflows, data pipelines
ReflexionMulti-Trial ReflectionHigh (Multiple episodes)Code generation, competitive programming
Tool Calling APINative Token GrammarLowestOpenAI / Anthropic native tool use
+-----------------------+-----------------------+-----------------------+
| Feature               | Chain-of-Thought (CoT)| ReAct Loop            |
+-----------------------+-----------------------+-----------------------+
| External Tools        | No                    | Yes                   |
| Grounding Feedback    | None (Pure Internal)  | Real-Time Environment |
| Latency               | 1 Inference Pass      | N Inference Turns     |
| Self-Correction       | Weak                  | High (via Observations|
| Determinism           | Low                   | Verified by Tools     |
+-----------------------+-----------------------+-----------------------+

When to use it — and when not to

Diagram: when the ReAct loop is the right structure

Choose the ReAct Loop when:

Do NOT use the ReAct Loop when:

The AI connection

Knowledge check

  1. What are the three core steps in a ReAct loop? Reason (Thought), Act (Action tool execution), and Observe (capturing environment feedback).
  2. How does an agent recover from a malformed JSON tool call? The runtime catches the parsing exception and injects the error back into the scratchpad as an Observation, enabling the agent to re-emit valid syntax.
  3. What is the primary difference between ReAct and Reflexion? ReAct operates within a single episode loop, while Reflexion evaluates entire completed episodes, generates verbal critiques, and updates episodic memory across multiple attempts.

Hands-on exercise

In this lab, you will implement a Production ReAct Execution Engine with Token Stream Parsing and Self-Correction in Python. Your implementation will:

  1. Construct a regex token stream parser that extracts Thought:, Action: tool(json_args), and Final Answer:.
  2. Build an isolated Tool Execution Dispatcher handling math, knowledge search, and string manipulation.
  3. Implement self-correcting error handling for invalid tool names and malformed JSON payloads.
  4. Execute simulated multi-step reasoning trajectories and verify termination predicates against unit tests.

Expected output

============================= test session starts ==============================
collected 5 items

tests/test_react_engine.py .....                                         [100%]

============================== 5 passed in 0.04s ===============================
[REACT ENGINE] Goal: "Find population of Seattle and calculate 20% growth"
[STEP 1] THOUGHT: I need to query Seattle's population.
         ACTION: search_kb({"query": "Seattle population"})
         OBSERVATION: "Seattle population: 750,000"
[STEP 2] THOUGHT: Now calculating 750000 * 1.20
         ACTION: calculate({"expr": "750000 * 1.20"})
         OBSERVATION: "900000.0"
[STEP 3] THOUGHT: I have all required data.
         FINAL_ANSWER: "The projected population is 900,000."
[SUMMARY] Completed in 3 steps with 100% test validation.

Validate your work

Run the pytest test suite:

pytest tests/run_tests.sh

Troubleshooting

  1. Regex parsing fails on multi-line JSON: Ensure your regex uses the re.DOTALL flag when capturing arguments.
  2. Tool execution parameter mismatch: Unpack dictionary arguments with **args safely or validate against expected signatures.

Common mistakes

Practice assignment

Add a Plan-and-Solve Preprocessor that generates an initial 3-step checklist and marks items as completed after each observation.

Extension challenge

Implement a Sliding Window Scratchpad Compactor that retains only the last 3 turns in full detail while replacing earlier turns with an automated 1-sentence summary.

Quiz

Q1. What is the primary advantage of interleaved Thought-Action-Observation (ReAct) over standalone Chain-of-Thought (CoT)?

  1. ReAct requires zero tokens
  2. ReAct eliminates all model non-determinism
  3. ReAct compiles Python code into assembly
  4. ReAct grounds reasoning in real observations
Show answer

Answer: D. ReAct grounds reasoning in real observations

Standard CoT cannot interact with external tools or correct internal reasoning errors using live observation feedback.

Q2. How should an agent runtime respond when an action parser fails to parse malformed JSON produced by the model?

  1. Feed the error and schema back as an observation
  2. Crash the application immediately
  3. Silently execute a random function
  4. Delete the agent's history
Show answer

Answer: A. Feed the error and schema back as an observation

Self-healing error feedback prompts the model to correct its syntax mistake on the subsequent reasoning turn.

Q3. What role does the 'Thought' phase serve in the ReAct prompt trajectory?

  1. It encrypts the user prompt
  2. It gives verbal working memory for planning
  3. It bypasses API rate limits
  4. It downloads the external dependencies it needs first
Show answer

Answer: B. It gives verbal working memory for planning

The Thought phase acts as an explicit scratchpad for step-by-step reasoning and sub-goal tracking.

Q4. What indicates to the ReAct engine that the task has successfully concluded?

  1. The HTTP socket closes
  2. The model runs out of GPU memory
  3. It emits Final Answer: or calls terminate()
  4. The Python garbage collector runs
Show answer

Answer: C. It emits Final Answer: or calls terminate()

A formal termination predicate (such as `Final Answer:` or a completion tool call) ends the loop.

Q5. What happens if an agent loop does not implement an observation feedback step?

  1. It becomes a closed-loop system
  2. It converts into a neural network
  3. It executes 100x faster with 100% accuracy
  4. It goes open-loop, blind to tool output
Show answer

Answer: D. It goes open-loop, blind to tool output

Without observation feedback, the agent cannot see what its tools produced or whether errors occurred.

Glossary

ReAct Paradigm
An execution pattern that interleaves verbal reasoning (Thought), tool execution (Action), and environment feedback (Observation).
Action Parser
A parser extracting structured tool names and JSON parameters from raw LLM output text.
Self-Healing Feedback
Injecting tool syntax or runtime errors back into the agent context so the model can correct its mistake.
Scratchpad
The cumulative working memory containing all historical thoughts, actions, and observations for the active trajectory.
Termination Predicate
The logical condition determining when an agent has satisfied its objective and should stop execution.
Plan-and-Solve
An agent architecture that generates a full execution plan upfront before executing individual steps.
Observation Injection
Appending structured tool returns to the active conversation history before the next reasoning step.
Stream Parsing
Incrementally evaluating model token streams to trigger tool calls with minimal latency.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.