How AI context compression actually works, part 1

Every request your coding agent sends is mostly things you've already paid for. The file it read three turns ago. The build log it scrolled past. The 500-row JSON blob a tool returned when the model needed four fields. All of it rides along on every single request, and all of it is billed again.

Context compression is the fix, and it's worth being precise about what it is, because the word invites a misunderstanding.

Compression here does not mean summarizing. No model paraphrases your context into something shorter and hopes the meaning survives.

Compression means the same information in fewer tokens: selecting what matters verbatim, re-encoding what's repetitive, and pointing back to what's already there. Densifying, not discarding.

Here's how that actually works, technique by technique.

Route first, compress second

The first step isn't compression at all, it's recognition. A local ML classifier looks at each block of content and decides what it is: a JSON payload, a build log, source code, a diff, prose. That takes a few milliseconds, on your machine.

This matters because there is no universal compressor. What's safe to do to a build log, collapse 400 lines of Compiling... into one, would be vandalism on source code. So each content type is routed to a compressor built for its shape, and anything the router isn't confident about passes through untouched.

This is also why honest benchmark tables have zeros in them. Grep results that are already dense, and source code the model actually needs to see: the router looks at both and does nothing. Doing nothing is a feature.

Repetitive structure: the JSON problem

Tool outputs love to return arrays of near-identical objects. A hundred log events, five hundred search results, every one carrying the same keys and mostly the same values.

A structure-aware compressor analyzes the array statistically: which fields vary, which are constant, where the values change. Then it keeps the head, the tail, the change points, and drops the verbatim duplicates in between. Every item it keeps is an untouched original, same schema, same bytes, no generated text mixed in.

Before, 500 items:

[
  {"level": "info", "service": "api", "msg": "request ok", "status": 200},
  {"level": "info", "service": "api", "msg": "request ok", "status": 200},
  ... 496 more, nearly identical ...
  {"level": "error", "service": "api", "msg": "upstream timeout", "status": 504}
]

After: the first few, the last few, and that one error, because that's where the information was. Measured on a real 500-item payload: 9,526 tokens → 1,614, in 2 milliseconds.

Logs: keep the signal, fold the noise

Build and test output is the most compressible thing an agent reads, and the most dangerous to compress naively. The rule: errors, warnings, and stack traces survive in full, including chained exceptions, where the interesting frame is often three Caused by: blocks deep. What gets folded is the repetition around them.

   Compiling serde v1.0.203
   Compiling tokio v1.38.0
   ... 180 lines like this ...
error[E0308]: mismatched types
  --> src/pipeline.rs:214:9

The error block comes through byte-for-byte. The 180 compile lines become a note that they happened. Measured: a 200-line build log went from 2,412 tokens to 148.

Lossless re-encoding: same facts, fewer bytes

Some transforms don't select at all, they re-encode, with an exact inverse. Strip ANSI color codes. Collapse a line repeated 40 times into the line plus a count. Group grep hits under one file heading instead of repeating the path on every match:

src/auth.rs:12: fn validate_token(
src/auth.rs:48: fn validate_token_expiry(
src/auth.rs:91: // validate_token is called from...

becomes

src/auth.rs:
  12: fn validate_token(
  48: fn validate_token_expiry(
  91: // validate_token is called from...

Grep stays grep, diffs stay diffs, logs stay logs, the model reads them exactly as it would have. And each of these transforms is self-checking: if reversing it doesn't reproduce the original, or the result isn't actually smaller, the content ships unchanged.

Déjà vu: the same file, four times

Here's the redundancy no per-block compressor can see. Your agent runs cat pipeline.py. Two turns later, git diff shows most of the same lines. Then a test failure prints twenty of them again. The same bytes, billed three times, on every request from then on.

Cross-turn deduplication watches the whole conversation. When a later tool output contains a large span that already appeared verbatim earlier, the later copy becomes a compact pointer: this content appeared in full above. The first occurrence is never touched, so the original is always physically in the context the model reads. And it's done carefully enough that earlier turns are never rewritten, which keeps provider-side prompt caching working, so the savings stack instead of fighting each other.

Nothing is ever truly gone

The last piece is the safety net under all of the above: reversibility. When compression does set something aside, the middle rows of that array, the folded stretch of log, it's kept, locally, and the model gets a way to ask for it. If the model decides it genuinely needs row 247, it retrieves row 247. In practice it rarely asks, because the compressors keep what mattered. But the option is what makes the whole pipeline trustworthy: compression that can be undone is compression you don't have to think about.

This is what runs on your Mac

Everything above runs locally, inside Compresso, on every request your agent makes, before the request is billed. No prompts leave your machine, and the parts that don't compress safely simply don't compress. Every plan includes the full pipeline; the tiers only mirror the Claude or ChatGPT plan you're already on.

If you want to see the technique-by-technique numbers, including the ones where compression does nothing at all, they're on the benchmarks page. Or just try it on today's session: five-day trial, no card up front.