Harness Context Window Limits and Long Task Degradation

Context window limits cause two distinct failures in coding agents, namely irreversible information loss at the hard cutoff and declining reasoning quality before the window fills. Reliable long-horizon work therefore depends on the model, the harness, and the execution environment. Replicas runs Claude Code, Codex, Cursor, and Opencode in isolated, pre-configured Linux VMs, providing cloud execution while leaving harness choice open.

Why context windows fail differently than most teams expect

Most teams budget for the moment an agent reaches its context limit, but the more consequential failure starts earlier. The hard cutoff forces the system to truncate or summarize its history, and both operations destroy information irreversibly. A root-cause hypothesis or constraint from earlier in the task can disappear. Soft degradation, by contrast, weakens recall and attention to early information while the transcript still fits inside the window. Available capacity is therefore not proof that the agent can still reason effectively over its contents.

LLM-as-Code names three common symptoms of long-horizon failure: token explosion, control-flow hallucination, and unreliable completion. The transcript grows without a structural bound, the agent can declare success before verification runs, and a recent failing test can displace a correct earlier diagnosis. These symptoms do not all require a full window. An agent can keep working as it gets slower, costs more, and makes less sense, so soft failure is harder to spot than a capacity limit.

Why benchmarks hide the soft failure from teams evaluating harnesses

Standard single-turn and short-run benchmarks measure accuracy at the moment of a call, leaving reliability after the fiftieth tool call unmeasured. They therefore miss the accumulated pressure that matters during a multi-day refactor. Coding makes this gap especially important because the work depends on relationships among units as well as retrieving isolated facts. Losing track of an earlier relationship can hurt later reasoning even when the agent still remembers the details.

Jarmak’s August 2026 work treats evaluation and operation as a dependency chain in which weaknesses in task construction, execution environments, retrieval, or state management can invalidate downstream conclusions. For harness selection, that means evaluating sustained refactors and feature work rather than treating single-turn accuracy as a complete reliability measure. When an execution platform such as Replicas supports multiple harnesses, the useful comparison is how each handles accumulating history, lost context, and intermediate state. Claude Code, Codex, Cursor, and Opencode need not fail in the same way even when they use a capable model. If short benchmarks cannot expose those differences, the next evidence comes from examining the agent loop and its behavior over a long run.

How the LLM-as-orchestrator architecture produces token explosion as a structural consequence

The architectural mismatch is straightforward because looping, branching, and sequencing are deterministic work assigned to a probabilistic system. When the LLM owns scheduling, it must repeatedly reconstruct its position in the task from the conversation. Every earlier tool call, output, and reasoning trace goes into the history the model sees again. Context therefore grows with the number of steps multiplied by the average output size, and the architecture puts no cap on it before the hard limit.

LLM-as-Code argues that better prompts, larger windows, and smarter retries do not make this arrangement reliable by construction. Prompts improve the chance of following a procedure, but their gains remain model-specific. Larger windows hold more history but cost more and add latency, and recall can still get worse before they fill. Retries reduce per-step error without bounding error over a long run. Basic tool calls in Claude Code can often consume 50k+ tokens per subagent, illustrating how quickly execution history can become a substantial burden.

As history accumulates, later steps cost more to process while earlier content becomes harder to use. Teams running more cloud agents therefore need full attribution of runs, including how many tokens they use and which model they use. Replicas tracks attribution across harnesses, models, and credentials, so teams can account for execution as they look at how agents loop and branch. LLM-as-Code proposes a more fundamental alternative called Agentic Programming, in which the program owns control flow and invokes the model only for reasoning or generation. Its DAG-structured context depends on call depth.

What the harness is and why it — not the model — is where degradation is fought

The harness is the infrastructure around the model that manages long-running work. It controls what enters context, what survives outside it, how tasks are separated, and whether verification takes place. Its structural primitives include a filesystem for durable state and collaboration, code execution, a sandbox for isolation and verification, memory across sessions, and context management. These responsibilities put the harness at the point where degradation can be detected, slowed, and mitigated. A stronger model does not replace the system that supplies its working conditions.

Context engineering acts on those conditions through compaction, state offloading, and task isolation into sub-agents. Compaction reduces the active transcript, external storage preserves state beyond the window, and isolated tasks limit how much unrelated history reaches a call. Vercel removed 80% of its agent’s tools and got fewer steps, fewer tokens, and faster responses. That result shows how a smaller tool surface can reduce the work that generates context in the first place. The engineering target is not simply fitting more material into the window, but controlling why it enters.

Jarmak finds that apparent model failures often originate elsewhere in the system and that improvements at one layer may not reach end-to-end outcomes. This makes the harness a better unit of investigation than the model call alone. The coupling also runs in the other direction because models trained with particular harnesses can become overfitted to those designs. Harness choices can therefore outlast the immediate task. Their effects concern both present reliability and how easily the system can adapt later.

How specific harness design choices create different context failure profiles

Claude Code uses progressive disclosure to keep from loading material before it is needed. It exposes YAML front matter first, full SKILL.md instructions next, and reference files and resources on demand. That restraint matters because the base system prompt already consumes substantial space before tools, MCPs, compaction summaries, and project documents enter. Compaction triggers around 80–90% of the context limit and preserves recent interactions while potentially discarding earlier material. Pruning older command output works well for interactive debugging, but puts multi-step plans at risk when they rely on something the agent saw earlier.

Codex makes a different part of the problem explicit through its agent loop and reusable planning artifacts. OpenAI’s long-horizon practice guide introduces Plan.md, Implement.md, and Documentation.md as structured ways to externalize state. Their role is to move task information out of an ever-growing conversation and into durable artifacts. This changes where continuity resides. It gives the harness a source of task state beyond the transcript.

Cursor’s compaction choices likewise determine what remains available on a given turn. Its session persistence has a default SQLite store and an optional append-only JSONL store, but a crash mid-turn may leave no session file. Persistence can support resumption after a restart, but lost state still means degraded context. Cursor’s Builds, shipped August 13, addresses a different source of overhead by starting cloud agents in pre-configured environments. Its Router, shipped July 22, adds model-selection policy by cost, balance, or intelligence, with model allow/block lists restricted to Enterprise admins.

Opencode is an open-source agent whose configurable harness gives teams control over context strategy while requiring decisions that more opinionated harnesses make automatically. That establishes a tradeoff in responsibility. Sema Code proposes another architectural option by separating the core agent engine from client layers and publishing it as a standalone library. Multi-tenant engine isolation is one of its mechanisms.

How "model drift" after many steps differs from context exhaustion, and why both need harness-level detection

Context exhaustion concerns window capacity, while model drift concerns instruction adherence and reasoning coherence. A model can drift long before it exhausts the window. For example, the latest failing test can persuade an agent to abandon a correct earlier diagnosis. The old diagnosis may still be present, but its presence does not guarantee that the agent will use it. Likewise, a transcript that says verification must run does not prove that it ran.

This distinction changes both detection and response. Token counts can expose capacity pressure, but they cannot establish that an agent followed its instructions or preserved a sound line of reasoning. Compaction controls how much context is active, while correction gates control whether execution should move forward. A small leaderboard difference measures neither of these properties over a sustained run. Benchmark position and drift resistance are independent enough that a leading model may still go off track after many steps.

Jarmak’s reliability framework includes a versioned catalog of records and an evidence ledger connecting claims to support. Its dependency-chain view calls for harness-level observability to show where reliability breaks down. Structured logging must track tool call patterns, instruction-adherence signals, and intermediate reasoning traces as well as token counts. Those records provide evidence of what the loop actually did. Exhaustion and drift can occur together, but treating them as separate signatures keeps capacity management from being mistaken for behavioral verification.

The harness design principles that keep long-horizon tasks coherent

A coherent long-horizon harness needs four structural commitments, namely bounded context per call, durable external state, task decomposition that contains accumulation, and verification before advancement. Bounded context changes the unit of work from the entire conversation to the information needed for a particular call. LLM-as-Code’s DAG approach demonstrates this by making context depend on call depth. Sub-agents and isolated task scopes follow the same general principle when they contain accumulated history. Durable state then preserves the task information that should not depend on any one window.

Task decomposition is strongest when the program, rather than the model, governs the execution path. The model can choose how to reason or generate within a step while leaving decisions about whether a required branch or check happens to the program. This separates judgment from procedure. OpenAI’s planning artifacts work alongside this approach to keep plans and implementation state outside the transcript. Together, bounded calls and external state reduce the need to reconstruct the whole task on every turn.

The agent must then pass verification before moving forward. Jarmak develops execution-based evaluation and correction gates in which failed runs change the next attempt and repeated trials become a release test. These gates address the failure in which an agent declares completion before the verifier runs. Mitchell Hashimoto’s six-stage adoption path likewise culminates in harness engineering, with mistakes converted into AGENTS.md instructions or purpose-built tools. Over three months of fixes to his harness, far fewer of his commits needed correction.

Those fixes should add as little as the task allows. A rule or tool can address a known failure without turning every limitation of the current model into a permanent architectural assumption. This matters because models and harnesses evolve together, and some scaffolding becomes unnecessary as models improve. Independently replaceable components make that adjustment easier. The harness best positioned to survive model updates is therefore the one with the fewest baked-in assumptions.

How can teams run coding agents in the cloud without managing their own infrastructure?

Cloud execution changes which context strategies are practical, but isolation alone does not solve degradation. A bare VM still requires setup work that consumes the agent’s attention and window budget. A pre-configured environment lets the agent begin execution and verification without that overhead. Cursor’s Builds illustrates this co-design by keeping failed builds from becoming active while agents continue using the last successful build.

Replicas provides another execution option by running Claude Code, Codex, Cursor, and Opencode in individual, isolated Linux VMs pre-loaded with the team’s dependencies and tooling. In each sandbox, an agent can install packages, run services, drive a browser, and check its work. Those capabilities support the real execution required to distinguish completed work from a claim of completion. They also provide the execution layer in which compaction, state offloading, and task isolation can be put into practice. The environment supports these strategies without replacing the harness logic that governs them.

For a bring-your-own API key cloud coding agent runner, credential support is a separate selection criterion from isolation and harness availability. Replicas’s full analytics attribute every minute of agent execution to the source, person, harness, model, credential, skills, and MCP servers. That attribution supports operational investigation, but drift detection still requires evidence from the agent loop. Reliable long-horizon work combines bounded context, durable state, controlled execution, and real verification, while harness-agnostic infrastructure lets teams change those strategies without rebuilding the execution environment.