AI work has shifted from clever prompts to engineered systems. This playbook explains the five-layer stack—prompt, context, harness, loop, graph—when to climb each layer, how to diagnose bottlenecks, and how to avoid cost blowups by prioritizing verification, shared state, and controlled orchestration over one-off model calls.
The center of a modern AI application is still a model, but the leverage has moved outward. Teams that once tuned prompts now engineer the system surrounding the model: they curate what it sees, govern how it runs, add verifiers that define “good,” and, when necessary, wire multiple specialists into a coordinated workflow. This layered shift isn’t hype—it reflects where failures and costs actually appear at scale. A perfect turn of phrase can’t supply missing facts; pristine inputs still drift over long runs; unverified loops can print garbage all night. If you’re feeling rising costs with flat outcomes, your bottleneck likely sits one layer out.
Think in five layers. Prompt engineering frames the single request. Context engineering decides what the model sees—retrieval, ranking, summarization, and editorial cuts. Harness engineering controls a full run: tools, state persistence, orchestration, evaluation, and recovery. Loop engineering removes you from the trigger by defining goals, verifiers, and stop conditions. Graph engineering coordinates multiple specialist agents with edges, conditional routing, parallelism, and shared state. Each layer wraps the previous; nothing becomes obsolete. The scarce skill migrates outward: from phrasing, to information architecture, to control design, to verification design, to decomposition and routing across a graph.
Use symptoms to choose your next investment. If the model misreads the ask, tighten the prompt contract. If answers lack facts or drift off-topic, fix retrieval and context compression. If long workflows wobble or fail silently, strengthen the harness and add evaluators. If you are still the human scheduler, specify loop triggers and verifiers. If one agent muddles distinct jobs—research, write, critique—split work into nodes, define a shared state object, and route by clear conditions. Measurable KPIs anchor each step: answerability, tool success rates, run completion and recovery, loop precision/recall, and graph-level throughput and cost per accepted artifact.
Adoption should be incremental. Start by codifying output contracts and adding light evaluators to your harness. Graduate to loop engineering once evaluators catch more defects than humans. Move to graph engineering only when specialization plus routing beats a single well-verified loop on quality, speed, and unit economics. Favor small, testable control graphs, explicit state schemas, and sandboxed tools. Treat verifiers as first-class code with coverage, drift checks, and budget caps. That path keeps your model costs predictable, your defect rate observable, and your agent fleet aligned to business outcomes rather than tokens burned.
The Five Layers and Why Leverage Moved Outward
Each layer engineers further from the model: prompt (the request), context (the window), harness (the run), loop (repetition with a verifier), graph (specialists plus routing). The shift is economic. Failures closer to production—run recovery, silent drift, wrong handoffs—are expensive, so control moves outward. The result is less obsession over phrasing and more attention to information architecture, state design, and evaluation that gates acceptance.
Two corollaries matter: a weak inner layer poisons everything outside it, and scarce talent migrates outward. In 2023, prompting was the differentiator. In 2026, decomposition, routing, and verification separate toy demos from production systems.
Context and Harness: From Good Inputs to Controlled Runs
Context engineering decides what the model sees at call time. Prioritize retrieval quality (freshness, authority), ranking signals (task relevance, novelty), and compression strategies (quoting vs. summarizing) tied to output contracts. Instrument answerability and factual coverage metrics; if coverage is low, fix sources or compression before touching prompts.
Harness engineering controls the run: tool access, step sequencing, state persistence, evaluators, and recovery paths. Treat tools as capabilities with least-privilege sandboxes, enforce idempotent steps, snapshot state between steps, and implement retry/backoff with budget caps. Add lightweight evaluators early—schema conformity, invariants, PII/policy checks—so defects surface before they become loops or graphs of defects.
Loop Engineering: Verifiers, Stop Conditions, and Cost Control
A loop runs until a condition holds. The generator is cheap; the verifier decides value. Design verifiers as code, not vibes: explicit acceptance tests, multi-signal scoring (accuracy, policy, style), and thresholds mapped to business KPIs. Choose loop archetypes deliberately—open-ended with periodic checks, closed with strict stop conditions, or nested loops for iterative refinement—then monitor precision/recall against human judgments to prevent quiet, costly drift.
Failure modes are economic: weak verifiers mint plausible junk at scale. Counter with budget guards (per-run and per-artifact caps), progressive evaluation (cheap heuristics gate expensive checks), and anomaly alerts when retries or token usage spike. When verifiers consistently outperform humans on defect catch, you’re ready to scale frequency or step up to graphs.
Graph Engineering: Nodes, Edges, and Shared State in Practice
Use graphs when the work splits into distinct specialties. Define nodes (researcher, writer, critic) with their own models, tools, and local verifiers. Specify edges for routing: conditional branches, fan-out for parallel steps, fan-in for joins, and back-edges for retries. The glue is shared state: a typed object that carries goals, artifacts, decisions, and evaluation scores across nodes so downstream steps inherit upstream context intentionally, not implicitly.
Start small: a three-node pipeline with explicit state schema, parallelizable steps, and a final gatekeeper verifier. Choose frameworks that make state and control explicit and observable. Add observability per edge (latency, success, token spend), per node (tool error rates), and global SLOs (accepted artifacts per dollar). Promote nodes only when they earn their keep in quality or throughput.
