AI Product NewsAgent InfrastructureEditor's Pick

Harness²: Google’s Open-Source Framework for Self‑Improving AI Agents

Google released Harness², an Apache‑2.0 framework that lets agents rewrite their prompts, guardrails, memory and tool guidance around a fixed model. This piece explains how Harness² applies two levels of recursion to improve agent behavior without fine‑tuning, what developers can practically adopt, and the measured benchmark gains.

NexusAI Editorial DeskOct 20, 20261.5K views8 min read

Harness²: Google’s Open-Source Framework for Self‑Improving AI Agents

AI Brief

Harness² is a practical, open‑source framework for recursive agent harnessing: it optimizes the system around a fixed foundation model — prompts, guardrails, memory schemas, tools, subagents, skills and plugins — by proposing edits, probing behavior on the same task, and composing evidence‑backed refinements. Google reports meaningful gains across WorkBuddy‑Bench, JobBench and LAB with Gemini 3.5 Flash and GLM‑5, and the project supports OpenCode/Codex executors. For developers and enterprise teams, Harness² offers a way to make coding agents and assistants self‑improving at the harness layer without expensive model retraining; the trick is instrumented probing, robust composition, and persistent lesson storage.

Harness² reframes agent improvement as a systems problem rather than a model‑update problem. Instead of altering model weights, the framework iteratively edits the agent’s harness — the collection of system prompts, guardrails, memory schemas, tool guidance, subagents and scripts that shape behavior — and uses contrastive execution to accept only edits that demonstrably improve outcomes on the same task. That approach fits enterprise constraints where models are fixed for compliance, cost, or certification reasons, while still enabling step‑wise behavioral improvement through reproducible, testable harness edits.

Practically, Harness² executes a Propose–Probe–Compose loop: editors propose candidate harness changes, the harness runs the same task multiple times to probe behavioral differences, and a composer synthesizes a new harness that keeps changes empirically supported by observed outputs. The system implements two recursion levels. Task‑level recursion focuses on the immediate job (parallel proposals or sequential refinement steps controlled by a --k budget). Harnessing‑level recursion persistently records adopted harnesses and lessons so future tasks begin with a stronger starting harness, enabling transfer without explicit supervised labels.

Benchmark results reported by the project show nontrivial improvements using fixed models: single Harness² steps improved GLM‑5 and Gemini 3.5 Flash across WorkBuddy‑Bench, JobBench and LAB, with gains up to roughly 11.5 points on WorkBuddy, 4.5 on JobBench and nearly 12 percentage points on LAB’s partial pass metric. The framework supports OpenCode and Codex‑style executors, parallel and sequential recursion modes, and practical tooling for local dry‑runs and judged benchmark evaluation. These gains demonstrate harness optimization can be a high‑leverage engineering path when model changes are impractical.

For developers building coding agents, enterprise assistants or autonomous workflows, Harness² offers concrete operational mechanics: define editable harness components, craft deterministic editors for candidate edits, instrument robust probes (repeatable task executions and judges), and maintain a persistent improvement harness that captures evidence‑backed lessons. The repository includes demo flows that run without model credentials, plus instructions for Vertex AI, Codex executors and Dockerized benchmark setups. The Apache‑2.0 license makes integration and redistribution straightforward for teams that need a permissive license.

Key Takeaways

Improve agents without weight updates

Harness² demonstrates that optimizing prompts, guardrails, memory and tool guidance can produce measurable performance gains on professional benchmarks without retraining foundation models—useful where models are fixed for cost, compliance or governance reasons.

Make edits testable and persistent

Use the Propose–Probe–Compose loop with deterministic probes and a persistent improvement harness so only evidence‑backed edits are adopted and future tasks benefit from prior lessons.

Treat harness changes like code

Manage harness edits through CI, review, and staged rollout; instrument judges and sandbox executors to avoid overfitting to evaluation artifacts or introducing data‑leakage and security gaps.

What Harness² Actually Changes

Harness² centralizes eight editable components that together shape agent behavior: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts and plugins. Each component changes the way context is constructed or how tools and delegation behave inside the agent loop. That means teams can target specific failure modes — inconsistent tool usage, insufficient memory recall, or brittle prompt instructions — by proposing targeted edits to the relevant harness component and validating those edits through controlled probes rather than broader, riskier changes to the model.

Two Levels of Recursion: Task vs. Harnessing‑Level

The framework’s core novelty is its two stacked recursion loops. Task‑level recursion (controlled by the --k budget) runs Propose–Probe–Compose on the current job: either in parallel to explore diverse candidate harnesses or sequentially to iteratively refine a single chain of improvements. Harnessing‑level recursion persists adopted harnesses and the evidence that supported them; these lessons become priors for subsequent task‑level refinements. For teams, this separation enables immediate task optimization and gradual cross‑task learning, balancing exploration and accumulation without changing model parameters.

Benchmarks, Models and Practical Gains

Google reports reproducible gains across three professional benchmarks — WorkBuddy‑Bench (office, web, code), JobBench (professional work), and LAB (legal tasks) — when Harness² refines harnesses for Gemini 3.5 Flash and GLM‑5. These improvements were measured without weight updates, which is important for applied teams that can’t or won’t retrain models. The release includes tooling to run benchmarked experiments, judge results, and resume runs. Empirical improvement on accepted benchmarks gives teams a defensible path to performance uplift without model procurement or costly fine‑tuning.

Developer Experience and Integrations

Harness² ships demo flows and setup guidance for local runs and cloud‑backed benchmarks. It supports OpenCode and Codex executors and provides instructions for Vertex AI integration, Docker for WorkBuddy, and dependencies like Pandoc for LAB. The API and command‑line tools let teams run dry‑runs, probe candidate edits deterministically, and compose adopted harnesses into a persistent directory. For product teams, this means they can add harness optimization into CI, instrument probes for regression checks, and manage harness changes through standard code review and deployment processes.

Risks, Limitations and Operational Guardrails

Harness‑level learning is powerful but not risk‑free. Because composition is driven by observed behavior rather than ground‑truth labels, probe judges and sandboxing must be carefully designed to avoid overfitting to evaluation quirks, leaking sensitive data during parallel probes, or amplifying undesirable behaviors through repeated edits. The repository warns against unrestricted network access for executors and recommends configuring judges and network restrictions. Operationally, teams should treat adopted harnesses like configuration changes: require code review, staged rollout, and monitoring to ensure behavior amplifies intended outcomes.

Frequently Asked Questions

How does Harness² avoid changing the underlying model?

Harness² works exclusively at the harness level: it proposes edits to prompts, guardrails, memory formats, tool interfaces and scripts rather than modifying model weights. Proposals are validated by running the same task multiple times and keeping only the compositions supported by improved behavior, so model parameters remain untouched while behavior changes through orchestration and context engineering.

Can I run Harness² with proprietary or hosted models?

Yes. The framework supports OpenCode and Codex executors and includes guidance for integrating hosted models (for example via Vertex AI). The core loop does not require model retraining, but you must ensure API access, cost management, sandboxing and compliance policies are set before running parallel probes or benchmark evaluations.

What is a practical --k (recursion) budget for production use?

Start small: use --k=1 to validate the Propose–Probe–Compose workflow and confirm judges and probes are reliable. For deeper improvements, run parallel mode with modest K (3–5) to explore candidates, or sequential mode with incremental K to refine a single harness. Monitor marginal gains versus API costs and adopt only changes with consistent, judged improvements.