AI Engineering: Agents and Applications › AI Coding Agents › Day 304
Hands-on lab — Day 304: Effective Agentic Coding Workflows
- ← Back to the Day 304 lesson
- Open the hands-on files on GitHub — clone or download them from the public labs repository
- Local path in your clone:
labs/sections/ai-engineering/day-304-effective-agentic-coding-workflows/
Commands
Setup
pip install -r requirements/requirements.txt Run
python examples/self_healing_runner.py Test
bash tests/run_tests.sh File tree
examples/self_healing_runner.py expected-output/examples-run.txt expected-output/FIELDS.md expected-output/measured-values.txt expected-output/run-output.txt expected-output/starter-run.txt expected-output/test-output.txt expected-output/test-run.txt metadata.yml README.md requirements/requirements.txt security.md starter/self_healing_runner.py tests/run_tests.sh tests/test_self_healing.py troubleshooting.md
Lab README
Day 304 Lab: Effective Agentic Coding Workflows
Lesson
- Lesson title: Effective Agentic Coding Workflows
- Day number: 304 of 365
- Lesson article: https://ai-roadmap-365.github.io/day-304-effective-agentic-coding-workflows
- Lab files: everything you need is in this directory — follow “How to run” below.
- Browse the course locally: from the repository root, this lab also appears in the course website at
/labs/day-304-effective-agentic-coding-workflowswhen the site is running.
Purpose
Build an Automated Self-Healing Test Runner and Diagnostic Parser in Python that captures test execution results, extracts stack trace diagnostics, and compiles repair prompts.
Learning objectives
- Execute shell test commands and capture multi-stream outputs.
- Parse Python tracebacks to extract failing files, lines, and error types.
- Format structured repair prompts for coding agents.
- Enforce max iteration safety limits.
Prerequisites
- Python 3.10+ installed
- pytest installed
Supported operating systems
- macOS, Linux, Windows WSL2
Hardware requirements
- Standard CPU, 512MB RAM
Required software
- Python 3.10+, pytest
Free and open-source options
- Python Standard Library, Pytest
Installation
python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements/requirements.txt
File structure
starter/self_healing_runner.py: Starter implementation skeletonexamples/self_healing_runner.py: Verified reference implementationtests/test_self_healing.py: Test suiteexpected-output/: Captured execution logs
How to run
python3 examples/self_healing_runner.py
What the commands do
- Executes test command, parses failing traceback, and formats diagnostic repair prompt.
Expected output
Execution output: Tests OK
Validation steps
Run test runner:
bash tests/run_tests.sh
Tests
The test suite validates:
- Passing command execution and exit code 0
- Failing command execution and exit code != 0
- Traceback parser extracting file, line, and exception message
- Structured repair prompt compilation
Cleanup
rm -rf .pytest_cache __pycache__ .venv
Troubleshooting
Verify regex matches Python standard traceback format.
Security notes
Runs completely offline on local CPU.
Extension exercises
Add support for pytest summary parsing (FAILED tests/test_x.py::test_y).
Navigation
Day number: 304 of 365
Expected output
FIELDS.md
# Measured Values
- `tracebacks_parsed`: 1
- `repair_prompts_generated`: 1
examples-run.txt
Execution output: Tests OK
measured-values.txt
tracebacks_parsed=1
repair_prompts_generated=1
run-output.txt
Execution output: Tests OK
starter-run.txt
An untouched starter fails the suite. Captured from a real run:
self = <examples.self_healing_runner.SelfHealingTestRunner object at 0x10abd1e00>
diagnostics = {'failing_file': 'auth.py', 'line_number': 15, 'error_type': 'ValueError', 'error_message': 'Invalid token'}
raw_output = 'Raw log output'
def format_repair_prompt(self, diagnostics: Dict[str, Any], raw_output: str) -> str:
> raise NotImplementedError('TASK 3: implement format_repair_prompt.')
E NotImplementedError: TASK 3: implement format_repair_prompt.
examples/self_healing_runner.py:25: NotImplementedError
=========================== short test summary info ============================
FAILED tests/test_self_healing.py::test_execute_passing_command - NotImplemen...
FAILED tests/test_self_healing.py::test_execute_failing_command - NotImplemen...
FAILED tests/test_self_healing.py::test_extract_traceback_diagnostics - NotIm...
FAILED tests/test_self_healing.py::test_format_repair_prompt - NotImplemented...
============================== 4 failed in 0.03s ===============================
Implement each TASK in starter/, copy it over examples/, and the same
suite goes green.
test-output.txt
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-304-effective-agentic-coding-workflows
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 4 items
tests/test_self_healing.py::test_execute_passing_command PASSED [ 25%]
tests/test_self_healing.py::test_execute_failing_command PASSED [ 50%]
tests/test_self_healing.py::test_extract_traceback_diagnostics PASSED [ 75%]
tests/test_self_healing.py::test_format_repair_prompt PASSED [100%]
============================== 4 passed in 0.09s ===============================
test-run.txt
============================= test session starts ==============================
platform darwin -- Python 3.14.0, pytest-9.1.1, pluggy-1.6.0 -- <repo>/.venv-tools/bin/python3.14
cachedir: .pytest_cache
rootdir: <repo>/labs/sections/ai-engineering/day-304-effective-agentic-coding-workflows
plugins: cov-7.1.0, anyio-4.14.2
collecting ... collected 4 items
tests/test_self_healing.py::test_execute_passing_command PASSED [ 25%]
tests/test_self_healing.py::test_execute_failing_command PASSED [ 50%]
tests/test_self_healing.py::test_extract_traceback_diagnostics PASSED [ 75%]
tests/test_self_healing.py::test_format_repair_prompt PASSED [100%]
============================== 4 passed in 0.09s ===============================
Source files
examples/self_healing_runner.py (2430 bytes)
import subprocess
import re
from typing import Dict, Any
class SelfHealingTestRunner:
def __init__(self, workspace_root: str, max_iterations: int = 4):
self.workspace_root = workspace_root
self.max_iterations = max_iterations
def execute_test_command(self, test_cmd: str) -> Dict[str, Any]:
result = subprocess.run(
test_cmd,
shell=True,
cwd=self.workspace_root,
capture_output=True,
text=True
)
return {
"exit_code": result.returncode,
"passed": result.returncode == 0,
"stdout": result.stdout,
"stderr": result.stderr,
"combined_output": result.stdout + "\n" + result.stderr
}
def extract_traceback_diagnostics(self, raw_output: str) -> Dict[str, Any]:
diagnostics = {
"failing_file": None,
"line_number": None,
"error_type": "UnknownError",
"error_message": ""
}
file_match = re.search(r'File "([^"]+\.py)", line (\d+)', raw_output)
if file_match:
diagnostics["failing_file"] = file_match.group(1)
diagnostics["line_number"] = int(file_match.group(2))
error_match = re.search(r'([A-Za-z]+Error|Exception): (.+)', raw_output)
if error_match:
diagnostics["error_type"] = error_match.group(1)
diagnostics["error_message"] = error_match.group(2).strip()
return diagnostics
def format_repair_prompt(self, diagnostics: Dict[str, Any], raw_output: str) -> str:
return (
f"# TEST FAILURE DETECTED\n"
f"Failing File: {diagnostics.get('failing_file')}\n"
f"Line Number: {diagnostics.get('line_number')}\n"
f"Error: {diagnostics.get('error_type')}: {diagnostics.get('error_message')}\n\n"
f"## Full Terminal Output:\n```text\n{raw_output.strip()}\n```\n\n"
f"## Instructions:\n"
f"1. Analyze the root cause of this failure.\n"
f"2. Apply a targeted search-and-replace fix.\n"
f"3. Do not modify unrelated functions."
)
if __name__ == "__main__":
runner = SelfHealingTestRunner(".")
res = runner.execute_test_command("python3 -c 'print(\"Tests OK\")'")
print("Execution output:", res["stdout"].strip())
metadata.yml (406 bytes)
lesson_id: D304
day: 304
kind: python
languages:
- python
setup_commands:
- pip install -r requirements/requirements.txt
run_commands:
- python examples/self_healing_runner.py
test_commands:
- bash tests/run_tests.sh
cleanup_commands:
- rm -rf __pycache__ .pytest_cache
requires_network: false
requires_api_key: false
estimated_minutes: 45
last_executed: '2026-08-30'
executed_on: macOS 15.0 Apple Silicon
requirements/requirements.txt (14 bytes)
pytest>=7.0.0
starter/self_healing_runner.py (1199 bytes)
"""STARTER -- implement each TASK below, then run the tests.
Every function that raises NotImplementedError is yours to write. The
imports, constants and data structures are given; the logic is not.
Run `bash tests/run_tests.sh` to see which tasks are still outstanding.
"""
import subprocess
import re
from typing import Dict, Any
class SelfHealingTestRunner:
def __init__(self, workspace_root: str, max_iterations: int=4):
self.workspace_root = workspace_root
self.max_iterations = max_iterations
def execute_test_command(self, test_cmd: str) -> Dict[str, Any]:
raise NotImplementedError('TASK 1: implement execute_test_command.')
def extract_traceback_diagnostics(self, raw_output: str) -> Dict[str, Any]:
raise NotImplementedError('TASK 2: implement extract_traceback_diagnostics.')
def format_repair_prompt(self, diagnostics: Dict[str, Any], raw_output: str) -> str:
raise NotImplementedError('TASK 3: implement format_repair_prompt.')
if __name__ == '__main__':
runner = SelfHealingTestRunner('.')
res = runner.execute_test_command('python3 -c \'print("Tests OK")\'')
print('Execution output:', res['stdout'].strip())
tests/run_tests.sh (253 bytes)
#!/usr/bin/env bash
set -e
export PYTHONPATH=examples:.
if [ -n "$PYTEST" ]; then
$PYTEST tests/test_*.py -v
elif command -v pytest &> /dev/null; then
pytest tests/test_*.py -v
else
python3 -m unittest discover -s tests -p 'test_*.py' -v
fi
tests/test_self_healing.py (1632 bytes)
import pytest
from examples.self_healing_runner import SelfHealingTestRunner
def test_execute_passing_command():
runner = SelfHealingTestRunner(".")
res = runner.execute_test_command("python3 -c 'print(\"Passed\")'")
assert res["exit_code"] == 0
assert res["passed"] is True
assert "Passed" in res["stdout"]
def test_execute_failing_command():
runner = SelfHealingTestRunner(".")
res = runner.execute_test_command("python3 -c 'assert False, \"Failed Assertion\"'")
assert res["exit_code"] != 0
assert res["passed"] is False
assert "AssertionError" in res["combined_output"]
def test_extract_traceback_diagnostics():
sample_trace = """
Traceback (most recent call last):
File "calculator.py", line 42, in divide
return a / b
ZeroDivisionError: division by zero
"""
runner = SelfHealingTestRunner(".")
diag = runner.extract_traceback_diagnostics(sample_trace)
assert diag["failing_file"] == "calculator.py"
assert diag["line_number"] == 42
assert diag["error_type"] == "ZeroDivisionError"
assert diag["error_message"] == "division by zero"
def test_format_repair_prompt():
runner = SelfHealingTestRunner(".")
diag = {
"failing_file": "auth.py",
"line_number": 15,
"error_type": "ValueError",
"error_message": "Invalid token"
}
prompt = runner.format_repair_prompt(diag, "Raw log output")
assert "# TEST FAILURE DETECTED" in prompt
assert "Failing File: auth.py" in prompt
assert "Line Number: 15" in prompt
assert "ValueError: Invalid token" in prompt
assert "## Instructions:" in prompt
Troubleshooting
Troubleshooting - Day 304
Verify test command exit codes are properly evaluated.
Security notes
Security - Day 304
Ensure subshell commands run in safe workspace environments.