How Compaction Works in Coding Agents (Pi, Droid)

Most coding agents eventually show some version of:

Compacting conversation...

At a glance, this article covers:

• why coding agents compact context in the first place
• what gets preserved as raw history, compressed memory, or deterministic harness state
• how Pi stores user intent, constraints, progress, decisions, next steps, and file state
• how Factory Droid uses anchored iterative summarization instead of repeatedly re-summarizing the whole past
• why repeated compaction can distort memory over time
• what good compaction should optimize for

A long-running coding agent does not keep an infinite transcript of everything that happened. Files were read, commands ran, tests failed, hypotheses were formed and discarded, requirements changed, and edits were made under assumptions that later turned out to be wrong. Eventually, all of that stops fitting inside the model's context window, so something has to go.

The interesting part is deciding what goes.

After compaction, the model is no longer operating from the conversation exactly as it happened. It is operating from a reconstructed working context:

system instructions
+ tools
+ project instructions
+ compressed history
+ structured harness state
+ recent uncompressed messages

That compressed history might be a human-readable summary. Some pieces might be reconstructed from disk. Others might be tracked deterministically by the harness. In OpenAI's Responses API, part of the prior state can even be carried through an opaque encrypted compaction item rather than a readable summary.

This makes compaction more than a context-window optimization. It is a working-memory policy. The harness decides what the model remembers exactly, what it remembers approximately, what it can recover later, and what disappears.

That policy matters when comparing agents such as Pi, Factory Droid, Claude Code, Codex-style agents, and custom internal agents. Two agents can use equally capable models and still behave very differently three hours into the same task, because their harnesses are constructing very different versions of the past.

Before compaction: what is actually inside a coding agent's context?

A simplified coding-agent conversation might look like this:

System prompt

Tool definitions

AGENTS.md / CLAUDE.md / project instructions

User:
Fix the authentication bug.

Assistant:
I'll inspect the auth path.

Tool call:
read src/auth/controller.ts

Tool result:
...

Assistant:
The problem may be in session creation.

Tool call:
read src/session/store.ts

Tool result:
...

User:
Also make sure we do not change the public API.

Tool call:
edit ...

Test output:
...

The next request normally contains much of what came before it. If $m_i$ represents a message or tool event, the conversation state after $n$ events looks approximately like:

The conversation is only part of the prompt. There is also fixed or semi-fixed overhead:

where $I$ is system and harness instructions, $T$ is tool definitions, $M$ is persistent project or memory context, $H_n$ is conversation history, and $R$ is space reserved for the model's next response.

For a model with usable context budget $W$, eventually:

At that point the harness can stop, delete old messages, retrieve old information only when necessary, or transform old history into a smaller representation. That last option is compaction.

The context window filling

The first thing to notice is that compaction does not usually compress everything. Most systems preserve a recent suffix of the conversation exactly, while older history is collapsed into a smaller representation.

That gives the agent two memory resolutions: older history becomes a lower-resolution lossy summary, while recent history remains high-resolution raw context. This recency bias is deliberate. The latest tool result, failed test, or user correction is usually more likely to matter on the next turn than a search result from two hours ago.

Why compact at all when models have huge context windows?

The obvious answer is that every context window is still finite, but that is only part of the story.

Long context is not necessarily equivalent to useful context. The Lost in the Middle line of work showed that model behavior can depend strongly on where relevant evidence appears inside a long prompt, with information in the middle often used less reliably than information near the beginning or end.

This is one reason a million-token model does not automatically imply that the best harness should continuously feed it 999,000 tokens.

There is also cost and latency. Every retained token may be processed again on later requests, subject to whatever prompt caching the provider supports.

Factory makes this point explicitly in its compression work: the objective should not be minimizing tokens per request, but tokens per completed task. If aggressive compaction causes the agent to forget something and then re-read five files, re-run tests, and rediscover a rejected approach, the apparent token saving was not actually a saving.

This turns context management into an optimization problem.

Compaction as lossy compression

A useful information-theoretic framing is to treat the old conversation as $H$ and the compact representation as $C$:

Usually:

Unlike gzip, though, we cannot reconstruct $H$ exactly from $C$. Compaction is lossy compression.

The real goal is not simply:

A zero-token summary achieves that perfectly and is useless. The real objective is closer to:

where $\mathrm{TaskLoss}$ measures how much worse the agent becomes because it is operating from $C$ instead of $H$.

For coding agents, the relevant information is not evenly distributed through the transcript.

Consider these pieces of history:

We opened package.json.

We tried approach A.

Approach A failed because the API guarantees ordering.

Do not change the public interface.

src/cache/client.ts was modified.

The remaining failing test is refresh_token_expiry.

Great, thanks.

They have wildly different future value.

A good compactor needs to estimate that value before knowing exactly what the agent will need later. That is the difficult part.

The simplest compaction algorithm

Imagine a conversation:

A basic strategy chooses a cut point k.

Messages before k are summarized:

Messages after k remain intact:

The next model sees:

instead of:

That gives us two forms of memory:

OLDER HISTORY
      ↓
 lossy summary

RECENT HISTORY
      ↓
 exact messages

But there are several design decisions hiding inside those lines.

Where should k be? Can it cut in the middle of a tool call? How large can S become? What happens during the second compaction? Are files tracked in the summary or somewhere else? Does the summarizer see complete tool outputs? What happens to the prompt cache?

Pi gives us unusually good visibility into these questions.

Pi: compaction as an inspectable harness primitive

Pi is useful because the compaction mechanism is documented and exposed through its source.

Pi automatically compacts when:

contextTokens > contextWindow - reserveTokens

The current default reserveTokens in the docs is 16,384, leaving capacity for the next model response. Compaction can also be triggered manually with /compact, optionally with instructions about what the summary should focus on.

That threshold is not simply:

context == model maximum

The harness needs output headroom too.

Pi first decides what not to compress

When compaction begins, Pi walks backward from the newest message until it accumulates roughly keepRecentTokens. The current documented default is 20,000 tokens.

Everything after that boundary remains raw. Everything before it becomes summarization material.

This:
              cut point
                  ↓
┌─────────────────┬──────────────────┐
│ OLD HISTORY     │ RECENT HISTORY   │
│ summarize       │ preserve exactly │
└─────────────────┴──────────────────┘

This is effectively a recency prior encoded directly into the harness.

The newest interactions are presumed likely to matter next, so they get the expensive treatment: exact preservation. Older interactions are converted into lower-resolution memory.

Pi does not cut anywhere it wants

A conversation is not just text. Tool calls create structure. If a compactor cut between a tool invocation and its result, the remaining context could contain a test failure without the command that produced it:

assistant:
I'll run the tests.

tool_call:
pytest

---------------- CUT ----------------

tool_result:
FAILED tests/test_auth.py

Pi explicitly avoids this class of cut. Tool results are not valid cut points; they stay associated with their tool invocation. This is a small implementation detail with a large implication: agent traces have semantics that ordinary chat transcripts do not. A compactor that treats every message independently can destroy those semantics.

What if one agent turn is enormous?

Coding agents can produce very large turns. One user instruction may cause a chain of reads, edits, test runs, failures, more reads, and more edits before the user speaks again:

assistant
→ read
→ result
→ read
→ result
→ edit
→ result
→ test
→ result
→ inspect another file
→ edit again
→ test again

There may not be a convenient user-message boundary near Pi's desired cut point. Pi calls this a split turn. If one turn exceeds the recent-context budget, Pi can preserve only the latter portion of that turn while separately summarizing the earlier prefix, so the retained suffix still makes sense.

Conceptually, the next context contains a history summary, a summary of the beginning of the current oversized turn, and the raw end of that turn. Without the turn-prefix summary, the retained suffix might say, "That confirms our hypothesis. I'll change it now," while the evidence that established the hypothesis has disappeared on the other side of the cut.

Pi changes the representation before asking for a summary

Pi does not send the historical conversation to the summarization model as though it were another normal chat session. It serializes the trace into labelled text:

[User]: Fix the login bug

[Assistant]: I'll inspect auth.ts

[Assistant tool calls]:
read(path="src/auth.ts")

[Tool result]:
...

[Assistant]:
The JWT generation looks correct...

The labels tell the compaction model that this is data to summarize, not a conversation to continue. But this preprocessing is itself a lossy boundary. Pi's docs state that tool results are truncated to 2,000 characters before summarization. If the important line appears after that cutoff, the summarizer cannot preserve it because it never saw it:

character 1
...
character 2000

[truncated]

character 8500:
ROOT CAUSE: migration 0187 was never applied

That means compaction quality is not determined only by the summarization model. It depends on what the harness selected, how it represented the trace, what the summarizer preserved, and how the next prompt reconstructs state from the result.

What Pi stores in a compaction summary

Pi's summary format is closer to a handoff document than a prose recap. The important move is that Pi does not ask for "a summary" in the abstract. It asks the compaction model to fill operational slots that matter for continuation:

## Goal / User Intent
what the user is trying to accomplish

## Constraints & Preferences
requirements, things the user forbade, style or API constraints

## Progress
what is done, in progress, blocked, or still failing

## Key Decisions
chosen approach, rejected alternatives, and rationale

## Next Steps
what the next agent turn should do

## Critical Context
facts that would be expensive or risky to rediscover

<read-files>
files inspected during the compacted span
</read-files>

<modified-files>
files changed during the compacted span
</modified-files>

That structure matters. Suppose the raw history says:

We looked at auth.ts.
JWT generation is correct.
The bug is actually in refreshSession().
Do not modify the public AuthClient interface.
We changed session-store.ts.
Two tests still fail.

A fluent but weak summary might say:

We investigated an authentication bug and made progress fixing the session handling implementation.

That is semantically reasonable and operationally terrible. The next agent turn needs the user constraint, the ruled-out hypothesis, the current target, the modified file, and the remaining failing tests:

Intent:
fix the authentication bug without changing the public API.

Decision:
JWT generation was ruled out.

Current target:
refreshSession().

Modified:
session-store.ts.

Remaining:
2 failing tests.

The summary format is therefore part of the agent architecture. It shapes what kinds of facts are likely to survive compaction.

Repeated compaction is where things get interesting

A single compaction is not the hard case. In a long session, the agent compacts, continues working, accumulates new history, and compacts again. After the first compaction, the summary might be represented as:

The agent then continues and generates more history, $H_2$. When the session reaches the threshold again, the memory state is updated rather than starting from a clean slate:

After another cycle:

Each new summary depends partly on the previous summary. If an early detail is missing or distorted in $S_1$, later summaries may carry that distortion forward.

Visual: repeated compaction

The visual below replays the key idea: raw events collapse into summaries, and after enough cycles the earliest tokens are no longer inspectable. The model can only use the latest compacted state.

Compaction drift

Suppose early in a debugging session the agent believes:

auth.ts is causing the bug.

Later it discovers:

auth.ts is correct.
session-store.ts is the real problem.

The raw conversation contains both beliefs, plus the evidence that invalidated the first one.

A good summary records:

Initial hypothesis:
auth.ts.

Status:
ruled out.

Root cause:
session-store.ts.

Evidence:
...

A bad summary might simply say:

Investigated an auth.ts issue related to session handling.

That flawed summary now becomes input to the next summary, so small distortions can become persistent state. Very loosely:

If $S_t$ already contains an error, the next update operates on a state containing that error. The summarizer does not necessarily know which part of $S_t$ was faithful and which part was distorted.

This is why repeated compaction should be evaluated differently from one-shot summarization.

Pi avoids asking the LLM to remember everything

Pi does not rely exclusively on prose summaries to remember files. Its compaction metadata includes cumulative readFiles and modifiedFiles. During later compactions, Pi combines file operations from newly compacted messages with file state carried by previous compaction or branch-summary entries.

This gives Pi two memory channels:

1. LLM-generated summary

This is the flexible, semantic memory. It carries the goal, decisions, progress, constraints, next steps, and the story of why the agent is doing what it is doing. It is useful because it can compress messy conversation into coherent intent. It is also lossy because the LLM has to choose what matters.

2. Deterministic harness state

This is the mechanically recorded memory. In Pi's case, the important examples are files read and files modified. It is narrower than a prose summary, but it is much more reliable for artifact facts because the harness can record those facts directly instead of asking the model to remember them.

That separation is important. If a fact can be derived mechanically, the harness should not rely on a probabilistic summarizer to remember it. File paths are the obvious example: if the agent read src/session/store.ts or modified tests/auth.test.ts, the harness can record that directly.

The same design pattern can apply to other operational facts: commands that ran, tests that failed, packages installed, migrations touched, URLs fetched, commits created, or unresolved TODOs. These are not things the model needs to phrase beautifully. They are state the harness can track and re-inject when needed.

Factory Droid uses anchored iterative summarization

Factory describes Droid's compression strategy as maintaining persistent conversation state instead of regenerating a full compressed history every time. Factory calls this anchored iterative summarization.

The central idea is simple:

Maintain a persistent structured summary. When compression is needed, summarize only the newly dropped span and merge it into the existing summary.

Imagine the message stream looks like this:

m1 m2 m3 m4 m5 m6 m7 m8 m9 m10

After the first compression, Droid might have:

S1 summarizes: m1..m4
recent raw messages: m5..m10

Later, more raw messages become old enough to drop:

S1 already summarizes: m1..m4
newly dropped span: m5..m7
recent raw messages: m8..m13

Droid does not need to summarize m1..m7 from scratch. It summarizes the delta and merges it into the persistent state:

Using Factory's anchor notation, the same idea can be written as:

The next prompt receives the updated summary plus the recent raw suffix:

The important idea is incremental maintenance. Droid is not repeatedly re-interpreting the entire past. It carries forward a structured memory and updates it with the new material that just got dropped.

Factory's fill line and drain line

Factory describes compression with two thresholds:

• $T_{max}$: the fill line where compression triggers.
• $T_{retained}$: the lower level the context falls back to after compression.

The context grows until it reaches $T_{max}$. Compression runs, the context drops toward $T_{retained}$, and then the cycle begins again.

If $T_{retained}$ is close to $T_{max}$, the system preserves more raw context but compacts more often. If $T_{retained}$ is much lower, compaction happens less often but removes a larger span of raw history each time. Factory calls out the practical consequences: narrow gaps increase summarization overhead and prompt-cache disruption, while wide gaps increase the risk of compressing away useful context.

Why tokens per task is a better objective

Suppose Agent A keeps a larger working context:

request 1: 80k
request 2: 85k
request 3: 90k

task complete

Agent B aggressively compacts:

request 1: 40k
compact

forgot earlier investigation
read 3 files again

request 2: 45k
compact

forgot why approach A was rejected
tries A again

request 3: 50k

reruns tests

request 4: 55k

Looking only at per-request context size makes Agent B look efficient, but total work tells another story. A more useful objective is:

total cost =
normal inference
+ compaction
+ cache misses
+ refetching
+ repeated work
+ quality loss

That is why Factory frames the target as minimizing tokens per task rather than tokens per request.

Factory evaluated the memory, not the summary

Factory's evaluation of context compression is interesting because it does not ask whether the summary resembles the original transcript. It asks whether the agent can still use the compressed context to continue the task.

The evaluation uses probes around four capabilities:

Responses are graded across dimensions such as factual accuracy, conversation state, artifact trail, completeness, continuity, and instruction following. This is the right target for coding agents because the summary is not the product. The next correct action is the product.

A compressed memory can be short, fluent, and still wrong if it causes the agent to edit the wrong file or repeat a rejected approach.

Artifact memory is still hard

One result from Factory's evaluation deserves special attention.

Artifact trail was the weakest dimension for all three compression approaches Factory compared. Factory scored 2.45/5, Anthropic 2.33, and OpenAI 2.19 in its reported experiment. Factory explicitly suggests that artifact preservation may need handling beyond generic summarization.

That connects directly to Pi's design.

Pi separately accumulates file-operation state rather than trusting the generated prose summary alone.

Factory's public material does not expose enough Droid implementation detail to say exactly which artifact facts are tracked deterministically inside Droid. The more useful point is the design question this raises:

Which parts of agent memory should be generated by an LLM, and which parts should be maintained mechanically by the harness?

That question goes beyond compaction. It applies to every part of agent memory: files touched, tests run, commands executed, tickets updated, branches created, and validation still pending.

Pi and Droid are closer than they first appear

Factory's original compression article contrasts its anchored iterative approach with a naive strategy that repeatedly summarizes the entire prefix. It would be easy to read that contrast as:

Droid = incremental memory
Pi    = ordinary summarization

That is not accurate for current Pi. The Pi docs describe prior compacted state being carried forward and newly compacted regions being incorporated into later summaries.

The useful comparison is not simply "iterative versus non-iterative." Better questions are:

• What recent suffix is retained verbatim?
• How is the cut point selected?
• What gets serialized for the summarizer?
• What state exists outside the LLM-written summary?
• How are artifacts preserved?
• How customizable is compaction?
• What gets re-injected after compaction?
• How is compaction evaluated?

Those are harness questions, and they explain why agents with similar models can feel different during long sessions.

Different agents expose different control surfaces

Pi makes compaction unusually hackable.

Extensions receive a session_before_compact event containing information such as the messages being summarized, previous summary, file operations, token counts, retained boundary, and whether the trigger came from a manual request, threshold crossing, or overflow recovery. An extension can cancel compaction or supply its own compacted state.

That means Pi can be used as an experimental environment for memory research.

Droid exposes a different layer of control. Current Droid settings expose the token threshold that triggers automatic compaction, per-model threshold overrides, and the model used to perform compaction. Droid also has a PreCompact hook that runs before manual or automatic compaction.

OpenAI's Responses API exposes another point in the design space. Its compaction can return an encrypted compaction item that carries forward key prior state using fewer tokens. The docs explicitly describe that item as opaque and not intended to be human-interpretable.

That creates a useful contrast:

Pi:
human-readable, inspectable summary

Factory Droid:
structured anchored memory plus lifecycle controls

OpenAI Responses:
opaque model-consumable compaction item

Human readability helps with debugging, inspection, portability, editing, and evaluation. Opaque state potentially gives the provider more freedom in how information is represented internally.

Neither choice is inherently correct for every system. They optimize for different things.

Compaction and prompt caching

Memory rewriting has another hidden cost: prompt-cache disruption. Prompt caching relies on repeated requests sharing an identical prefix.

Before compaction:

SYSTEM
TOOLS
A
B
C
D
E
F
NEW MESSAGE

the provider may have cached a long prefix.

After compaction:

SYSTEM
TOOLS
SUMMARY
E
F
NEW MESSAGE

the prefix changes at SUMMARY.

Even though E and F themselves did not change, they now occur after a different prefix and their previous cached computation cannot necessarily be reused.

Pi's engineering write-up explicitly calls out that compaction breaks the existing prompt-cache prefix, after which later requests begin building a reusable prefix again.

Factory similarly includes frequent cache invalidation among the downsides of overly narrow compression intervals.

So the cost model for compaction is actually:

total cost =
normal inference
+ summary generation
+ cache miss
+ rediscovery

Again, the smallest context is not automatically the cheapest agent.

Context compression is only one possible memory architecture

There is a broader connection here to systems such as MemGPT.

MemGPT framed long-running LLM memory using an operating-system analogy: a small working context acts like fast memory, while information can move between different memory tiers to create the appearance of much larger effective context.

Coding agents are increasingly converging on something similar.

They have something resembling:

L1: current raw context
    very fast
    expensive
    limited

L2: compacted working state
    smaller
    lossy

L3: project files / memory files / specs
    persistent
    reloadable

L4: searchable historical state
    large
    retrieved when necessary

This makes context window size a poor description of an agent's actual memory capacity.

The more useful question is:

What memory hierarchy does the harness implement?

The bigger point

The model does not have memory. At inference time, it has input, and the harness constructs that input.

For short tasks, this distinction is easy to ignore because most useful history still fits verbatim. For long-running agents, it becomes impossible to ignore. After enough time, the model may be operating from a summary of a summary of a summary, plus structured state maintained by the harness, plus a recent slice of raw conversation.

That means the harness gradually becomes the author of the model's past. It decides which abandoned hypotheses disappear, whether a user constraint survives, whether exact filenames remain available, whether an invoked skill is reloaded, whether the model sees an inspectable summary or an opaque compressed representation, and how much recent reality remains untouched.

This is why two coding agents running similarly capable models can feel very different after hours of work. The difference may not be the model. It may be what the harness allowed the model to remember.

I expect this to become more important, not less, as context windows grow. A million-token window delays the first compaction. It does not remove the memory problem.

References