AI Engineering: Agents and Applications › AI Coding Agents › Day 303

Day 303: Working with a Coding Agent

Day 303 of 365 — Working with a Coding Agent

Master human-agent collaboration patterns for software development: author high-leverage technical specifications, assemble surgical codebase context, guide incremental refactorings, and prevent agent drift.

Course
AI Engineering: Agents and Applications
Category
AI Coding Agents
Reading time
≈ 35 min
Practical time
≈ 50 min
Lesson duration
1h 25m
Last verified
2026-08-30

Hands-on lab for this lesson

Lab files on GitHub: https://github.com/ai-roadmap-365/ai-roadmap-365.github.io/tree/main/labs/sections/ai-engineering/day-303-working-with-a-coding-agent

  1. Get the hands-on files. Clone the labs repository once (you can reuse this clone for every lesson). This works on macOS, Linux, and Windows (PowerShell or WSL):
    git clone https://github.com/ai-roadmap-365/ai-roadmap-365.github.io.git
    cd ai-roadmap-365.github.io
  2. Open this lesson's lab. Move into the directory for this specific day. Every lab lives at the same predictable path — section / subsection / week / day:
    cd labs/sections/ai-engineering/day-303-working-with-a-coding-agent
  3. Read the lab guide. Open `README.md` in that directory. It lists the exact commands, what each does, the expected output, and how to check your work — read it before running anything.
  4. Run it and check your work. Follow the README's "How to run" section: run the example first to see the finished result, then complete the numbered exercises in `starter/`, then run the tests. The tests pass (exit 0) only when your work is correct.
    bash tests/run_tests.sh   # or the test command named in the lab README

You can also open the lab as a local page (works offline, shows the file tree and expected output).

Learning objectives

By the end of this lesson you will be able to:

Prerequisites

Why this matters

Pair programming with an autonomous AI coding agent is fundamentally different from searching Google, browsing StackOverflow, or chatting with a generic conversational chatbot. An AI coding agent has direct, read-write access to your local filesystem, compiler toolchains, and git repository history. If guided with vague or underspecified prompts, an agent can hallucinate non-existent libraries, delete critical comments, refactor untouched modules without permission (agent drift), or enter infinite loops attempting to fix broken tests.

Conversely, when guided by an engineer skilled in Specification-Driven Development, surgical context gathering, and active collaborative steering, a coding agent becomes a 10x force multiplier: churning through complex refactoring, implementing comprehensive unit test suites, and shipping well-architected features in minutes.

Mastering how to work with a coding agent—authoring unambiguous requirements, assembling high-signal context packages, enforcing non-goals, and executing corrective pivots—is the cornerstone skill of the modern AI software engineer.

Specification Driven Prompting Anatomy diagram showing Goal, Context Files, Type Signatures, Explicit Constraints, Non-Goals, and Verification Plan

The idea in plain language

Diagram: an ambiguous instruction compared with a bounded specification

Diagram: the five components of a specification a coding agent can execute

Think of working with a coding agent like directing an exceptionally fast, highly capable junior engineer who lacks institutional memory:

The quality of the output is a direct mathematical function of the clarity and boundaries established in your specification.

Historical background

Diagram: the software engineering practices agentic specification draws on

The methodologies governing human-agent collaborative programming emerged across three major paradigms:

1. Test-Driven Development (Kent Beck, 2002)

TDD established the principle of writing automated verification tests before implementation code (Red -> Green -> Refactor). In agentic coding, golden test cases serve as deterministic guardrails for language models.

2. Specification by Example (Gojko Adzic, 2011)

Adzic demonstrated that framing software requirements with concrete input/output examples eliminates ambiguity between product managers and engineers. Coding agents thrive on concrete JSON and type examples.

3. Prompt-Driven Software Engineering (2023–Present)

Modern research into agentic developer productivity revealed that dividing tasks into Planning, Surgical Context Gathering, Patch Application, and Automated Verification produces an order-of-magnitude reduction in code defects compared to one-shot code generation.

What it is — and what it is not

Let us define the core principles of effective human-agent collaboration:

What Working with an Agent Is:

What Working with an Agent Is Not:

+-------------------------------------------------------------------------------+
|                       SPECIFICATION PROMPTING COMPARISON                      |
+-------------------+---------------------------+-------------------------------+
| Attribute         | Naive "Vibe" Prompt       | High-Leverage Spec Prompt     |
+-------------------+---------------------------+-------------------------------+
| Goal Definition   | "Fix the auth bug"        | "Handle JWT expiry in AuthToken"|
| Context Scope     | Entire repo dump / none   | `auth/token.py`, `test_jwt.py`|
| Type Constraints  | Unspecified               | Pydantic `TokenPayload` model |
| Non-Goals         | None (Agent refactors all)| "Do not modify public API"    |
| Verification Gate | Developer manual click    | `pytest tests/test_jwt.py`    |
| Success Rate      | ~25%                      | ~95%                          |
+-------------------+---------------------------+-------------------------------+

Why it was created and what problems it solves

Specification-driven collaboration solves four critical agent failure modes:

1. Eliminating Context Pollution & Hallucinations

Passing 50 irrelevant files forces the transformer’s attention mechanism to distribute across thousands of distracting tokens. Providing surgical interfaces guarantees high-attention focus.

2. Preventing Unwanted Refactoring (“Agent Drift”)

Explicit Non-Goals (“Do NOT touch database.py”) keep the agent contained to the assigned module.

3. Rapid Convergence on Green Tests

By providing the exact test failure command, the agent uses the test runner’s traceback to diagnose its own errors in a closed loop.

4. Consistent Enterprise Code Standards

Prompt templates ensure that code written by AI matches corporate style guides, docstring formats, and error handling protocols.

How it works

Let us inspect the collaborative steering loop:

[1. Human Developer Authors Spec Prompt]
  - Objective: Add rate limiting to /v1/chat endpoint
  - Files: `api/routes.py`, `middleware/limiter.py`
  - Constraints: Use in-memory Redis token bucket, max 50 req/min
  - Verification: `pytest tests/test_limiter.py`
                         │
                         ▼
[2. Agent Emits Implementation Plan]
  - Plan: Create RedisTokenBucket class -> Wrap route in decorator -> Add unit test
                         │
                         ▼
[3. Developer Reviews and Approves Plan]
  - Developer: "Approved. Proceed with implementation."
                         │
                         ▼
[4. Agent Executes File Edits]
  - Modifies `middleware/limiter.py` and `api/routes.py`
                         │
                         ▼
[5. Agent Runs Test Verification Tool]
  - Executes: `pytest tests/test_limiter.py`
  - Result: 4 passed in 0.05s
                         │
                         ▼
[6. Human Developer Reviews Diff and Merges]

Human Agent Collaborative Steering Loop showing Developer Spec, Agent Plan Review, Code Patch Execution, Test Gate, and Steering Intervention

An everyday analogy

Diagram: an imprecise sketch compared with a toleranced specification for a machine

Think of working with a coding agent like authoring a precision blueprint for a CNC precision milling machine:

Examples in practice

Let us build a complete Python Specification Compiler and Context Bundler that compiles user requirements into structured agent prompt packages:

from typing import Dict, Any, List
import os

class SpecCompiler:
    def __init__(self, workspace_root: str):
        self.workspace_root = os.path.realpath(workspace_root)
        
    def bundle_context(self, relative_file_paths: List[str]) -> str:
        '''Extract and format file contents into clean XML/Markdown context blocks.'''
        blocks = []
        for path in relative_file_paths:
            abs_path = os.path.join(self.workspace_root, path)
            if os.path.exists(abs_path):
                with open(abs_path, "r", encoding="utf-8") as f:
                    content = f.read()
                blocks.append(f"### File: {path}
```python
`{content}`
```")
            else:
                blocks.append(f"### File: {path} (NOT FOUND)")
        return "

".join(blocks)
        
    def compile_prompt(
        self,
        goal: str,
        target_files: List[str],
        constraints: List[str],
        non_goals: List[str],
        verification_command: str
    ) -> str:
        '''Synthesize an unambiguous, high-leverage specification prompt.'''
        context_str = self.bundle_context(target_files)
        
        prompt_lines = [
            f"# TASK OBJECTIVE
{goal}
",
            f"# SURGICAL CONTEXT
{context_str}
",
            "# HARD CONSTRAINTS",
            "
".join([f"- {c}" for c in constraints]),
            "
# NON-GOALS (DO NOT MODIFY)",
            "
".join([f"- {ng}" for ng in non_goals]),
            f"
# VERIFICATION PLAN
Run the following command to verify your work:
`{verification_command}`
",
            "# INSTRUCTION FOR AGENT",
            "1. Output your step-by-step implementation plan first.",
            "2. Apply targeted edits using search-and-replace chunks.",
            "3. Run the verification command and report test results."
        ]
        return "
".join(prompt_lines)

if __name__ == "__main__":
    compiler = SpecCompiler(".")
    spec = compiler.compile_prompt(
        goal="Add rate limiting to API routes",
        target_files=["routes.py"],
        constraints=["Use token bucket algorithm", "Max 100 req/min"],
        non_goals=["Do not modify auth middleware", "Do not add third-party dependencies"],
        verification_command="pytest tests/test_limiter.py"
    )
    print("Compiled Spec Prompt Preview:
", spec[:300])

Implications: security, privacy, performance, scalability, and cost

1. Accidental Exposure of Secrets in Context Bundles

When bundling context files into agent prompts, developers frequently include .env files or API key credentials. Always configure prompt bundlers to ignore .env, .pem, and credentials files.

2. Context Window Cost vs Reasoning Precision

Including full source files for 10 modules can cost $0.15 per turn. Compiling surgical interface extracts (classes, methods, type hints) reduces per-turn cost to $< $0.01 while increasing reasoning accuracy.

Deep Dive: The Mathematics of Context Window Hygiene and Attention Dilution

Transformer language models rely on the Scaled Dot-Product Attention Mechanism: Attention(Q, K, V) = softmax(Q * K^T / sqrt(d_k)) * V

Diagram: how irrelevant context dilutes a transformer's attention distribution

When an engineer dumps 40 irrelevant source files (totaling 80,000 tokens) into an agent’s context window:

  1. Attention Entropy Dispersion: The softmax probability distribution across keys ($K$) flattens. The mathematical attention weight assigned to the critical 5-line bug location is diluted by orders of magnitude.
  2. Needle-in-a-Haystack Degradation: Models suffer from context degradation (the “lost-in-the-middle” effect), frequently overlooking critical constraints placed in the middle of massive context dumps.
  3. Surgical Context Principle: By pruning context down to the exact interface contracts, Pydantic type definitions, and failing stack trace lines (keeping context under 4,000 tokens), the attention weights concentrate directly on the target function, increasing reasoning accuracy from 32% to over 94%.

The Art of the Corrective Pivot Intervention

When supervising a coding agent, engineers must recognize the warning signs of Agent Cognitive Traps:

Diagram: the three failure loops a coding agent falls into

When an agent enters a cognitive trap, the engineer must immediately:

  1. Interrupt the agent execution loop.
  2. Issue a Corrective Pivot Prompt:
    • “STOP. Revert your last three edits to database.py.”
    • “The issue is not connection pooling; it is an unhandled None value in user.profile.avatar_url.”
    • “Implement a null-coalescing guard inside serializers/user.py:L24 only. Do NOT modify any other files.”
  3. Verify that the agent locks onto the corrected trajectory and executes the minimal diff.

Alternatives: free, open source, and commercial

ApproachToolingGranularityBest Suited For
Spec-Driven PromptingCustom Markdown TemplatesHigh (Surgical)Production refactorings & features
Interactive ComposerCursor / WindsurfMedium (Interactive)Rapid exploratory development
Autonomous Issue BotDevin / SWE-agentHigh (Async PR)Headless bug fixing from Jira/GitHub
Unconstrained Vibe PromptingRaw ChatLow (Noisy)Throwaway prototyping
+-----------------------+-----------------------+-----------------------+
| Dimension             | Ad-hoc Chat Prompt    | Spec-Driven Agent Prompt|
+-----------------------+-----------------------+-----------------------+
| Upfront Effort        | 10 seconds            | 2 minutes             |
| First-Turn Success    | < 30%                 | > 90%                 |
| Regression Risk       | High (Agent edits all)| Low (Enforced limits) |
| Verification Model    | Manual Inspection     | Automated Pytest Gate |
+-----------------------+-----------------------+-----------------------+

When to use it — and when not to

Diagram: when specification-driven prompting is warranted

Choose Specification-Driven Prompting when:

Do NOT require heavy specifications for:

Production Case Study: Spec-Driven Feature Refactoring at Scale

Let us examine a real-world enterprise engineering case study involving a payments processing service:

Diagram: the measured difference between ad-hoc prompting and a specification on one task

The Cognitive Burden of Prompt Ambiguity

When humans communicate with other humans, we rely on shared cultural context, verbal inflections, and body language to disambiguate vague requests. Large language models possess no physical presence or unstated institutional memory; they operate purely on the mathematical probability distribution of tokens conditioned on the provided prompt string.

Diagram: how an ambiguous requirement expands into competing implementations

When an engineer leaves a requirement ambiguous (e.g. “Handle errors properly”), the model must sample across hundreds of plausible error handling strategies:

By explicitly defining the error contract in your specification prompt (“Catch StripeCardError, log warning with customer ID, and raise HTTPException(status_code=402, detail=error.user_message)”), you collapse the model’s uncertainty, ensuring deterministic code synthesis.

Checklists for Production-Grade Specification Authoring

Before hitting Enter on a complex coding agent task, verify your prompt against the following Production Readiness Checklist:

  1. Target File Bounds: Are the exact relative file paths explicitly specified?
  2. Schema & Data Types: Are Pydantic / TypeScript type signatures included for all inputs and outputs?
  3. Hard Invariants: Are error codes, status returns, and edge cases clearly stated?
  4. Explicit Non-Goals: Does the prompt forbid modifying unrelated files, changing public signatures, or introducing third-party dependencies?
  5. Deterministic Verification Gate: Is an exact pytest or test command attached so the agent can self-verify completion?

Diagram: the five-point checklist for a production coding specification

Adhering to this 5-point checklist guarantees that the agent executes with surgical precision.

The AI connection

Knowledge check

  1. What are Non-Goals in a spec prompt? Explicit restrictions telling the agent what files, APIs, or architectures it must NOT modify.
  2. Why is surgical context bundling superior to whole-repository dumps? It prevents context pollution, reduces token costs, and focuses the model’s attention on key interfaces.
  3. What is the first step an agent should take before writing code? Output a clear implementation plan for human review.

Hands-on exercise

In this lab, you will implement a Specification Compiler and Prompt Bundler in Python. Your implementation will:

  1. Build a surgical context bundler extracting target file contents with line annotations.
  2. Compile structured specification prompts enforcing constraints, non-goals, and verification commands.
  3. Verify prompt structure and missing file handling across automated pytest tests.

Expected output

============================= test session starts ==============================
collected 5 items

tests/test_spec_compiler.py .....                                        [100%]

============================== 5 passed in 0.02s ===============================
[SPEC COMPILER] Bundled 2 context files (38 lines).
[PROMPT GENERATION] Formatted 3 constraints and 2 non-goals.
[VERIFICATION GATE] Attached test command: 'pytest tests/test_feature.py'.

Validate your work

Run the pytest test suite:

pytest tests/run_tests.sh

Troubleshooting

  1. Missing context file: Ensure relative paths match the workspace root directory structure.
  2. Malformed prompt sections: Verify newline separation between XML/Markdown heading blocks.

Common mistakes

Practice assignment

Add an AST Signature Extractor to the bundle_context method that extracts only class definitions and function headers if a file exceeds 200 lines.

Extension challenge

Implement a Specification Validator that parses the agent’s proposed plan using regular expressions or JSON schemas to verify all constraints are respected before granting execution permission.

Quiz

Q1. What is the primary cause of AI coding agents hallucinating APIs or producing broken implementations on complex codebases?

  1. The computer simply running out of disk space
  2. Using Python instead of JavaScript
  3. Vague prompts plus noisy, unfocused context
  4. Running tests in the background
Show answer

Answer: C. Vague prompts plus noisy, unfocused context

Context pollution and vague specifications are the primary drivers of agent hallucinations and incorrect assumptions.

Q2. What is 'Specification-Driven Development' when pairing with an AI coding agent?

  1. Allowing the model to guess the entire architecture without instructions
  2. Deploying directly to production without testing
  3. Writing code in assembly language first
  4. Write the spec before asking for the code
Show answer

Answer: D. Write the spec before asking for the code

Clear upfront specifications constrain the model's solution space and ensure high-precision code generation.

Q3. Which context-gathering technique provides the highest signal-to-noise ratio for a coding agent?

  1. The interface, the tests, and the stack trace
  2. Copying every file in the repository into the chat window
  3. Sending only the project README
  4. Providing raw git binary packfiles
Show answer

Answer: A. The interface, the tests, and the stack trace

Surgical context packages (signatures, tests, tracebacks) minimize token waste and maximize model attention on key interfaces.

Q4. What should a developer do when they observe a coding agent stuck in a repetitive loop (e.g. failing a test, making an incorrect edit, and failing the same test again)?

  1. Let it run for 1,000 turns unattended
  2. Interrupt, correct the root cause, and revert
  3. Format the hard drive
  4. Delete all existing unit tests
Show answer

Answer: B. Interrupt, correct the root cause, and revert

Active human steering breaks repetitive loops and resets the agent's reasoning trajectory.

Q5. Why is defining explicit 'Non-Goals' in a feature prompt valuable when directing an AI coding agent?

  1. It makes the prompt longer
  2. It disables all unit tests
  3. It forbids refactors and changes outside the task
  4. It forces the agent to compile C++ code
Show answer

Answer: C. It forbids refactors and changes outside the task

Non-goals prevent 'scope creep' and protect existing codebase architecture from unrequested refactoring.

Glossary

Specification-Driven Development
Directing coding agents with structured technical specs defining requirements, schemas, and non-goals.
Context Window Hygiene
Curating minimal, high-signal context to prevent attention dilution and prompt bloat in coding models.
Non-Goals
Explicit boundaries in a prompt specifying what the agent must NOT modify or refactor.
Agent Drift
When an agent wanders off the original task trajectory and begins making unrequested codebase modifications.
Surgical Context Package
A curated collection of relevant interface contracts, test cases, and error logs provided to an agent.
Turn-by-Turn Scoping
Breaking a complex multi-hour feature into discrete, verifiable 10-minute milestones for the agent.
Corrective Pivot Prompt
A targeted intervention message steering an agent out of an incorrect assumption or failing loop.
Golden Test Case
A deterministic unit test defining the exact expected behavior that the agent's code must satisfy.

Sources and further reading


Kept in this browser, no account needed. Your progress page turns the whole record into one link you can bookmark or open on another device.