# Build Your Own Agentic Coding Harness

An agentic coding harness is the control system around a language model. The model proposes the next move. The harness decides what context it sees, which tools exist, whether an action is allowed, how the workspace changes, when the run stops, and what counts as proof.

This guide shows how to build that control system so you can change the underlying model without rewriting the rest of the product.

> The honest compatibility claim: support any model that can reliably return text. Use native tool calling when a provider supports it. Use a constrained text protocol only as a degraded fallback. Model independence does not mean capability parity.

## The mental model

```text
USER TASK
    |
    v
RUN CONTROLLER ---> CONTEXT ENGINE ---> MODEL ADAPTER
    ^                                      |
    |                                      v
    |                              TEXT OR TOOL REQUEST
    |                                      |
    |                                      v
    +--- LEDGER <--- VERIFIER <--- POLICY + APPROVAL
                                      |
                                      v
                              SANDBOXED TOOL EXECUTOR
                                      |
                                      v
                              WORKSPACE + OBSERVATION
```

The model is one dependency. The harness is the product.

| The model owns | The harness owns |
| --- | --- |
| Generating text | Run state and stop conditions |
| Suggesting tool calls | Tool schemas and validation |
| Proposing code changes | Filesystem boundaries and patch application |
| Interpreting observations | Authorization and approval gates |
| Choosing a likely next step | Time, step, token, and cost budgets |
| Producing a candidate answer | Tests, verification, traces, and replay |

## 1. Start with a canonical model contract

Do not make the rest of the application understand every provider's message format. Normalize at the edge, but preserve capability differences explicitly.

```ts
export type HarnessMessage = {
  role: "system" | "user" | "assistant" | "tool";
  parts: Array<
    | { type: "text"; text: string }
    | { type: "tool_call"; id: string; name: string; args: unknown }
    | { type: "tool_result"; id: string; result: unknown; isError?: boolean }
  >;
};

export type ModelCapabilities = {
  nativeTools: boolean;
  parallelTools: boolean;
  structuredOutput: "strict" | "best_effort" | "none";
  streaming: boolean;
  maxInputTokens?: number;
  maxOutputTokens?: number;
};

export type ModelRequest = {
  messages: HarnessMessage[];
  tools: ToolDefinition[];
  maxOutputTokens: number;
  providerOptions?: Record<string, unknown>;
};

export type ModelEvent =
  | { type: "text_delta"; text: string }
  | { type: "tool_call"; id: string; name: string; args: unknown }
  | { type: "usage"; inputTokens?: number; outputTokens?: number }
  | { type: "done"; finishReason: string }
  | { type: "error"; error: HarnessError };

export interface ModelAdapter {
  id: string;
  capabilities(): Promise<ModelCapabilities>;
  generate(request: ModelRequest, signal: AbortSignal): AsyncIterable<ModelEvent>;
}
```

Build one adapter per provider or local runtime. The adapter translates provider-native events into `ModelEvent` and converts canonical tools back into the provider's expected schema.

### The adapter rules

1. Keep provider SDK logic inside the adapter.
2. Preserve tool-call IDs. They connect the request, execution receipt, and returned observation.
3. Expose capability flags instead of pretending every model behaves the same.
4. Retry transport failures in the adapter. Do not silently retry side-effecting tools.
5. Record the requested model and the resolved model version when the provider returns it.
6. Pass provider-specific features through `providerOptions`; do not contaminate the common contract.

OpenAI, Anthropic, and Gemini all implement the same broad application-side cycle: provide tool definitions, receive a structured request, execute it in your application, return the result, and continue. Their exact message shapes and schema guarantees differ, which is why the adapter boundary matters. See [OpenAI function calling](https://developers.openai.com/api/docs/guides/function-calling), [Anthropic tool use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview), and [Gemini function calling](https://ai.google.dev/gemini-api/docs/function-calling).

### Fallback for text-only models

If a model does not support native tools, ask it to emit exactly one action envelope:

```json
{
  "kind": "tool_call",
  "tool": "workspace.read_file",
  "arguments": { "path": "src/app.ts" }
}
```

Parse it, validate it, and reject anything outside the envelope. Label this adapter `structuredOutput: "best_effort"` or `"none"`. Never execute prose that merely looks like a command.

## 2. Make the run a deterministic state machine

The model should not own the loop. A persisted controller should.

```text
CREATED
  -> PREPARING_CONTEXT
  -> WAITING_FOR_MODEL
  -> REVIEWING_ACTION
  -> WAITING_FOR_APPROVAL   (only when required)
  -> EXECUTING_TOOL
  -> VERIFYING
  -> COMPLETED | FAILED | CANCELLED | BUDGET_EXHAUSTED
```

A run record needs enough information to pause, resume, replay, and explain itself:

```ts
export type Run = {
  id: string;
  task: string;
  constraints: string[];
  workspace: { root: string; revision: string; writableRoots: string[] };
  adapterId: string;
  model: string;
  status: RunStatus;
  step: number;
  budgets: { maxSteps: number; maxMs: number; maxTokens?: number; maxCostUsd?: number };
  approvals: ApprovalRecord[];
  startedAt: string;
  updatedAt: string;
};
```

Persist every transition before beginning the next side effect. Give every proposed tool call an idempotency key. A process crash should resume from a known state, not guess what happened.

### Stop conditions belong in code

Stop when any of these becomes true:

- The verifier accepts the completion claim.
- A hard budget is exhausted.
- The user cancels.
- A required approval is denied.
- The same recoverable failure repeats beyond its retry policy.
- The harness detects no progress, such as an unchanged diff across repeated write attempts.

## 3. Treat tools as typed capabilities

A tool is not a prompt snippet. It is a named capability with input and output contracts, a risk level, a timeout, and an executor.

```ts
export type ToolDefinition = {
  name: string;
  description: string;
  inputSchema: Record<string, unknown>;
  outputSchema?: Record<string, unknown>;
  risk: "read" | "write" | "execute" | "network" | "destructive";
  timeoutMs: number;
};

export interface HarnessTool {
  definition: ToolDefinition;
  execute(args: unknown, ctx: ToolContext): Promise<ToolResult>;
}
```

Use JSON Schema to validate both arguments and structured results. JSON Schema Draft 2020-12 is the current published version. The 2026-07-28 MCP tool specification also uses JSON Schema for `inputSchema` and optional `outputSchema`, and recommends timeouts plus audit logging. See [JSON Schema 2020-12](https://json-schema.org/draft/2020-12) and [MCP tools](https://modelcontextprotocol.io/specification/2026-07-28/server/tools).

### Begin with seven tools

| Tool | Purpose | Default risk |
| --- | --- | --- |
| `workspace.list` | Enumerate files under an allowed root | Read |
| `workspace.read_file` | Read a bounded text range | Read |
| `workspace.search` | Search filenames and text | Read |
| `workspace.apply_patch` | Apply a structured patch inside allowed roots | Write |
| `workspace.diff` | Return the current diff | Read |
| `shell.run` | Run an allowlisted command with timeout and output limits | Execute |
| `verify.run` | Run named lint, test, build, or policy checks | Execute |

Prefer narrow tools over a universal shell. A narrow tool is easier to authorize, validate, test, and explain.

### Keep MCP optional

MCP is a useful transport for discovering external tools and resources. It should plug into the tool registry, not become the internal architecture of the whole harness.

```text
BUILT-IN TOOLS ----\
PLUGIN TOOLS -------+--> TOOL REGISTRY --> POLICY --> EXECUTOR
MCP TOOLS ----------/
```

The current MCP core is stateless. If an external tool needs durable state, pass an explicit opaque handle and validate authorization on every call. See the [MCP 2026-07-28 release](https://blog.modelcontextprotocol.io/posts/2026-07-28/).

## 4. Put policy between intention and execution

Never send a model-generated action directly to an executor.

```ts
export type PolicyDecision =
  | { outcome: "allow"; ruleId: string }
  | { outcome: "require_approval"; ruleId: string; summary: string }
  | { outcome: "deny"; ruleId: string; reason: string };
```

Evaluate the proposed tool, validated arguments, workspace, user identity, current run, and prior approvals.

| Action | Suggested default |
| --- | --- |
| Read files inside the workspace | Allow |
| Search inside the workspace | Allow |
| Apply a patch inside declared writable roots | Allow, then show diff |
| Run a named local test or formatter | Allow with timeout |
| Install dependencies or access the network | Require approval |
| Write outside the workspace | Deny |
| Read secrets, credentials, or keychains | Deny |
| Delete data, force-push, deploy, or message people | Require explicit approval or deny |

Approvals must be specific. Store the tool name, normalized arguments, scope, expiry, user, and decision. An approval for `npm test` is not approval for arbitrary shell access.

The MCP specification recommends a human ability to deny tool invocations. OWASP guidance adds schema enforcement, least privilege, sandboxing, restricted network access, and audit trails. See [MCP tools: user interaction](https://modelcontextprotocol.io/specification/2026-07-28/server/tools#user-interaction-model) and [OWASP Agentic AI threats and mitigations](https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/).

## 5. Isolate the workspace and executor

Assume generated commands and retrieved content are untrusted.

The minimum safe execution boundary should provide:

- One explicit workspace root.
- A clean snapshot or branch before writes.
- Read-only mounts except for declared writable roots.
- No secrets by default.
- Network disabled by default.
- A non-root user.
- Per-command timeout, process-tree termination, and output limits.
- CPU, memory, disk, and process limits.
- A diff after every write phase.
- A disposable environment for generated code when risk warrants it.

Containers are useful isolation, but they are not automatically a security boundary. Choose containers, microVMs, WebAssembly, or a managed sandbox according to the consequence of escape. OWASP's [Securing Agentic Applications Guide](https://genai.owasp.org/download/49059/) discusses these isolation choices and the principle of least privilege.

### Patch, do not rewrite blindly

Use structured patches for code changes:

1. Read the target file and record its content hash.
2. Ask the model for a patch against that exact version.
3. Reject the patch if the file changed after the read.
4. Apply inside allowed roots only.
5. Return the resulting diff as the tool observation.
6. Let the verifier decide whether the change is acceptable.

This makes conflicts visible and creates a compact receipt for review.

## 6. Build context as a budgeted product

Do not dump a repository into the prompt. Assemble context in layers.

| Layer | Contents | Retention rule |
| --- | --- | --- |
| Policy | System rules, workspace boundary, approval rules | Never summarize away |
| Task | User goal, acceptance criteria, constraints | Never summarize away |
| Repository | Relevant instructions, file map, selected source | Refresh when files change |
| Working state | Current plan, changed files, open failures | Keep explicit |
| Recent evidence | Tool results, diffs, test output | Trim by relevance and age |
| History | Older dialogue and observations | Summarize with receipts |

Track where every context item came from. Treat repository files, tool outputs, search results, and issue text as data, not trusted instructions.

### Compaction invariants

A summary must retain:

- The user's exact task and constraints.
- The current workspace revision.
- Files changed and why.
- Commands run with exit codes.
- Unresolved test failures.
- Approval decisions and their scope.
- Tool receipts needed to replay the run.

If the summary cannot preserve those invariants, stop and ask for a larger context budget rather than fabricating continuity.

## 7. Implement the loop

```ts
while (!terminal(run.status)) {
  enforceBudgets(run);

  const context = await contextEngine.build(run.id);
  ledger.append("model.requested", summarizeRequest(context));

  const response = await collect(adapter.generate(context.request, abortSignal));
  ledger.append("model.responded", response.receipt);

  if (response.toolCalls.length === 0) {
    const verdict = await verifier.verifyCompletion(run, response.text);
    ledger.append("verification.finished", verdict);
    run.status = verdict.ok ? "COMPLETED" : "PREPARING_CONTEXT";
    continue;
  }

  for (const call of response.toolCalls) {
    const tool = registry.require(call.name);
    const args = schemas.validate(tool.definition.inputSchema, call.args);
    const decision = await policy.evaluate(run, tool.definition, args);
    ledger.append("tool.proposed", { call, decision });

    if (decision.outcome === "deny") {
      observations.add(toolDenied(call, decision.reason));
      continue;
    }

    if (decision.outcome === "require_approval") {
      const approval = await approvals.pauseAndWait(run, call, decision);
      if (!approval.granted) {
        observations.add(toolDenied(call, "User denied approval"));
        continue;
      }
    }

    const result = await executor.run(tool, args, run.workspace);
    schemas.validateOptional(tool.definition.outputSchema, result.structured);
    ledger.append("tool.finished", result.receipt);
    observations.add(toolObservation(call.id, result));
  }

  run.step += 1;
  run.status = "PREPARING_CONTEXT";
}
```

Important details:

- Collect and validate the full tool request before execution, even when the model streams it.
- If parallel tool calls are allowed, parallelize reads first. Serialize writes unless independence is proven.
- Send tool errors back as structured observations so the model can recover.
- Do not turn a model's final sentence into success without verification.

## 8. Record an append-only run ledger

The ledger is the source of truth for debugging and replay.

```text
run.created
context.built
model.requested
model.responded
tool.proposed
approval.requested
approval.decided
tool.started
tool.finished
verification.started
verification.finished
run.completed
```

Each event should include a timestamp, run ID, sequence number, event version, parent event, redacted payload or hash, and trace identifiers. Never log secrets or raw sensitive prompts by default.

For telemetry, align spans with the developing OpenTelemetry GenAI conventions: `invoke_agent`, model inference, `execute_tool`, and `plan` when you can identify an actual planning phase. See [OpenTelemetry GenAI agent spans](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-agent-spans.md).

### Replay has two modes

- **Deterministic replay:** reuse stored model and tool outputs to reproduce controller decisions.
- **Live replay:** rerun the same task against a selected model and compare outcomes.

Do not promise identical model text. Promise that the control decisions and evidence are inspectable.

## 9. Verify independently

The verifier should not ask the same model, "Did you finish?" and accept yes.

Use a pipeline of deterministic checks first:

1. The diff is inside the requested scope.
2. Required files exist.
3. Formatting, lint, typecheck, tests, and build return recorded exit codes.
4. No secret or policy scan regressed.
5. The user acceptance criteria map to explicit evidence.
6. The workspace is in a reviewable state.

Use a second model grader only for criteria that cannot be expressed deterministically, and keep its evidence separate.

```ts
type VerificationReport = {
  ok: boolean;
  checks: Array<{
    name: string;
    status: "pass" | "fail" | "skipped";
    evidence: string;
  }>;
  changedFiles: string[];
  unresolvedFailures: string[];
};
```

## 10. Build it in five increments

### Increment 1: Read-only investigator

- One provider adapter.
- `list`, `read_file`, and `search` tools.
- Run state, budgets, and a JSONL ledger.
- No writes and no shell.

**Exit test:** the agent answers a repository question and every claim links to a file and line range.

### Increment 2: Patch author

- Add structured patch application.
- Enforce workspace roots and content hashes.
- Show a diff after every write.

**Exit test:** a stale-file patch is rejected without modifying the workspace.

### Increment 3: Guarded executor

- Add named verification commands.
- Add timeouts, output limits, process cleanup, and approval gates.
- Disable network by default.

**Exit test:** an unapproved network or out-of-root action cannot run.

### Increment 4: Resumable runs

- Persist state transitions and approval requests.
- Add deterministic replay.
- Add cancellation and crash recovery.

**Exit test:** kill the process while waiting for approval, restart it, and resume exactly once.

### Increment 5: Provider matrix and evals

- Add at least two more adapters.
- Run the same fixture suite across providers.
- Compare task success, unsafe-action rate, tool recovery, cost, latency, and regression rate.

**Exit test:** changing one configuration value swaps the model without changing tools, policies, verification, or the fixture suite.

## Suggested repository structure

```text
coding-harness/
  src/
    core/
      types.ts
      controller.ts
      state-machine.ts
      budgets.ts
    adapters/
      model-adapter.ts
      openai-adapter.ts
      anthropic-adapter.ts
      gemini-adapter.ts
      text-protocol-adapter.ts
    context/
      builder.ts
      compactor.ts
      provenance.ts
    tools/
      registry.ts
      schemas.ts
      workspace-tools.ts
      shell-tool.ts
    policy/
      engine.ts
      approvals.ts
      rules.ts
    runtime/
      sandbox.ts
      process-runner.ts
      workspace.ts
    ledger/
      events.ts
      store.ts
      replay.ts
    verify/
      pipeline.ts
      checks.ts
    cli.ts
  fixtures/
    repositories/
    tasks/
    expected/
  tests/
    adapters/
    policy/
    recovery/
    evals/
```

## The evaluation suite

Create small seeded repositories and run the same tasks repeatedly.

### Normal cases

- Find and explain a bug without writing.
- Add a small function and its tests.
- Update a dependency within an allowed range.
- Refactor a module without changing behavior.

### Failure injections

- Malformed tool arguments.
- Unknown tool name.
- Model timeout or truncated stream.
- Patch conflict after a file changes.
- Tool timeout and oversized output.
- Test failure after a plausible patch.
- Approval denial.
- Network disabled.
- Process crash during approval wait.
- Prompt injection hidden in a repository file.

### Metrics

| Metric | What it proves |
| --- | --- |
| Task success rate | The requested result is actually achieved |
| Regression rate | Existing behavior remains intact |
| Unsafe action rate | Policy blocks forbidden effects |
| Approval precision | Users are interrupted for consequential actions, not routine reads |
| Tool recovery rate | The loop recovers from structured failures |
| False-completion rate | The verifier catches unsupported success claims |
| Replay completeness | Every control decision has enough evidence to reconstruct it |
| Cost and latency | Provider comparisons include operational tradeoffs |

## Definition of done

Your first credible harness is ready when:

- [ ] A model can be swapped through configuration.
- [ ] Capability differences are exposed, not hidden.
- [ ] Every tool input is schema-validated before execution.
- [ ] Every write stays inside declared roots.
- [ ] Every command has a timeout, exit code, output cap, and process cleanup.
- [ ] Network access and destructive actions require explicit policy decisions.
- [ ] Runs can pause for approval and resume once.
- [ ] Every action creates a redacted receipt.
- [ ] A deterministic verifier can reject a false completion.
- [ ] The fixture suite runs unchanged across at least three adapters.

## System proof rubric

| Criterion | Question |
| --- | --- |
| Task defined | Does the run carry explicit acceptance criteria and budgets? |
| Mechanism visible | Can a reviewer inspect context, decisions, tool calls, diffs, and traces? |
| Failure handled | Do denial, timeout, conflict, crash, and failed-test paths recover safely? |
| Reusable tomorrow | Can the same tools, policies, and evals run against another model? |

Only call it **Ready to ship** when all four have evidence attached.

## Primary references

- [OpenAI function calling](https://developers.openai.com/api/docs/guides/function-calling)
- [Anthropic tool use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview)
- [Gemini function calling](https://ai.google.dev/gemini-api/docs/function-calling)
- [Model Context Protocol 2026-07-28 tools](https://modelcontextprotocol.io/specification/2026-07-28/server/tools)
- [MCP 2026-07-28 release notes](https://blog.modelcontextprotocol.io/posts/2026-07-28/)
- [JSON Schema Draft 2020-12](https://json-schema.org/draft/2020-12)
- [OpenTelemetry GenAI agent spans](https://github.com/open-telemetry/semantic-conventions-genai/blob/main/docs/gen-ai/gen-ai-agent-spans.md)
- [OWASP Agentic AI threats and mitigations](https://genai.owasp.org/resource/agentic-ai-threats-and-mitigations/)
- [OWASP Securing Agentic Applications Guide](https://genai.owasp.org/download/49059/)
- [NIST Generative AI Profile](https://www.nist.gov/publications/artificial-intelligence-risk-management-framework-generative-artificial-intelligence)