ai-agent

A typed, self-evolving agent harness for tool-using LLMs.

This is a self-evolving agent: it can inspect the repository, make scoped improvements, run checks, and propose changes as pull requests.

Agent = Model + Harness
The model reasons. This repository is the harness: the loop, tools, memory, retrieval, guardrails, and multi-agent runtime that turn a chat completion into a reliable agent.

Built with Clean Architecture, strict typing (mypy), and a test suite that includes mutation testing (mutmut) — designed as both a production starting point and a hireable reference codebase.

Most “agent demos” hide the interesting part inside a framework black box. This kit makes the harness explicit and readable:

Everything below is deterministic software around the model (OpenRouter today — swap via LLMPort).

• Evolve artifacts live under .ai_agent/evolve/<run_id>/. | HarnessBank Gene Bank | Done (v1) | (where)×(why) cells + gated screening stubs; evolver ≠ task agent | features/harness_bank | | Organism worker (Phase 2) | Done | CI wait, MergePolicy, file scheduler, STOP kill switch | features/evolve/organism.py | | Ops metrics | Done (log) | JSONL OpsEvent, replay/compare hooks | harness/ops_metrics.py | | OS / container sandbox | — | Scoped limits only (not Docker/Firecracker) | Roadmap |

Model (not harness): whatever you set in YAML (openai/gpt-4o-mini, etc.) via OpenRouter.

src/ai_agent/
├── domain/           # models, ConversationState, Tool protocol, ports
├── harness/          # Agent, ReAct loop, ToolRegistry, guards
├── features/         # brief, self_harness, evolve, harness_bank
├── adapters/         # OpenRouter, SQLite, Chroma, DuckDuckGo, FS, git, gh, WebSocket
├── orchestration/    # multi-agent runtime + AgentFactory / spawn
├── tools/            # agent-facing tools over ports
├── support/          # shared helpers (console I/O)
└── entrypoints/      # CLI composition root

                  ┌─────────────────────────────────────────┐
  User / WS  ───► │  AgentRuntime (orchestration)           │
                  │    inbox → Agent.step → outbox / spawn  │
                  └───────────────┬─────────────────────────┘
                                  │
                  ┌───────────────▼─────────────────────────┐
                  │  Harness loop (application)             │
                  │  decide → ToolRegistry (gather) → obs   │
                  └───────┬─────────────────────┬───────────┘
                          │                     │
                 LLMPort (model)         tools → ports → infra
                 OpenRouter              search / FS / git / PR / RAG

Dependency rule: inward only. The loop depends on protocols, never on httpx/Chroma/SQLite directly. See DECISIONS.md. Marketing line: Agent = Model + Harness—and the harness can evolve under proof.

Requirements: Python 3.12+, an OpenRouter API key.

git clone https://github.com/pythonbyte/ai-agent.git
cd ai-agent

uv sync --all-extras
cp .env.example .env   # OPENROUTER_API_KEY=...

uv run ai-agent -c config/agent_config.yaml

With pip:

pip install -e ".[dev,rag]"
ai-agent -c config/agent_config.yaml

First product vertical on this harness: a research operator that turns a topic into a cited brief.

uv run ai-agent brief "agent harness"
# → briefs/20260731_agent-harness.md  (Summary / Key findings / Sources / Open questions)

uv run ai-agent -c config/agents/operator.yaml   # interactive

The operator prefers web_search → http_get → local retrieve / workspace_search, remembers prefs via memory (pref.* keys), and never invents sources. Optional --approve gates publication. Irreversible future actions use the request_approval tool.

Industry frontier (not “solved”): a fixed model improves the software around itself — prompts, tool descriptions, loop budgets — from execution evidence, without weight updates. Canonical loop: weakness mining → harness proposal → validation (held-in improves, held-out does not regress).

Reading path

Scaffold in this repo (human-gated): failures auto-log on step exceptions; harness propose writes a JSON patch (prompt append / max_tool_rounds only); harness accept runs a pytest subset then merges into YAML. No auto-merge. No arbitrary Python edits in v0.

uv run ai-agent harness record-failure "Timed out waiting for researcher"
uv run ai-agent harness propose
uv run ai-agent harness accept patch_… -c config/agents/operator.yaml

Upgrade path: HarnessBank Gene Bank + gated screening (harness-bank screen) and the Self-Evolving Engineer (evolve) for code PRs under PathPolicy. See HarnessBank (arXiv:2607.13683).

uv run ai-agent evolve "add PathPolicy unit tests"
# optional HITL: add --approve
# → .ai_agent/evolve/<run_id>/{run.json,plan.md,result.json}
# → branch + PR after local checks (human merges)

uv run ai-agent evolve-worker              # schedule next wake; respect STOP
uv run ai-agent evolve-worker --auto-merge # Phase 2: CI + MergePolicy only

Touch .ai_agent/evolve/STOP to halt forever-loops.

Enable tools by name in YAML — the registry resolves them at composition time.

model: openai/gpt-4o-mini
system_prompt: >
  You are a helpful assistant with tools. Prefer tools for facts and
  computation instead of guessing.
max_tool_rounds: 5
personality:
  tone: professional
  style: concise
greeting: "Hello! How can I help?"
workspace_root: "."
sqlite_path: ".ai_agent/state.db"
chroma_path: ".ai_agent/chroma"
tools:
  - calculator
  - run_python
  - web_search
  - http_get
  - workspace_search
  - memory
  - retrieve
compaction:
  enabled: true
  max_context_chars: 48000
  keep_recent_messages: 12
  max_summary_chars: 4000

Multi-agent personas live under config/agents/.

from ai_agent import BaseTool, ToolParameter, ToolResult

class EchoTool(BaseTool):
    def __init__(self) -> None:
        super().__init__(
            name="echo",
            description="Return text in UPPERCASE.",
            parameters=[
                ToolParameter(name="text", type="string", description="Text", required=True)
            ],
        )

    async def execute(self, arguments: dict[str, object]) -> ToolResult:
        text = str(arguments.get("text", ""))
        return ToolResult(tool_name=self.name, success=True, output=text.upper())

Register at the composition root (see examples/custom_tool.py).

Architecture:

Your terminal  --WebSocket-->  localhost:8765  -->  Docker (ai-agent --server)

The CLI console (ai-agent without --server) needs a TTY inside the container and is awkward for “type on host → reply from Docker.” Use the WebSocket server instead.

# 1) Ensure .env has OPENROUTER_API_KEY
cp -n .env.example .env

# 2) Start the agent server in Docker
docker compose up --build

# 3) From another terminal on the host, chat:
websocat ws://localhost:8765
# type a message, Enter → agent reply comes back as plain text

Without websocat:

# one-shot with Python
python - <<'PY'
import asyncio, websockets
async def main():
    async with websockets.connect("ws://localhost:8765") as ws:
        print(await ws.recv())          # greeting
        await ws.send("What is 2+2? Use tools if needed.")
        print(await ws.recv())
asyncio.run(main())
PY

Operator persona in Docker:

docker compose run --service-ports agent \
  ai-agent --server --host 0.0.0.0 --port 8765 -c config/agents/operator.yaml -v

Requirement: server must listen on 0.0.0.0 (compose already does). localhost inside the container would not accept host connections.

Tests mirror source packages under tests/ (domain/, harness/, features/, …). See AGENTS.md.

uv run pytest
uv run pytest --cov=src/ai_agent
uv run ruff check src tests
uv run ruff format src tests
uv run mypy src
uv run mutmut run      # high-value pure modules
uv run mutmut browse   # triage survivors

Survivors are expected on a first baseline (many are equivalent string/bound tweaks). Kill important ones with focused assertions; don’t gate CI on 100% without triage.

MIT © Eduardo Chiarotti (pythonbyte)