Can short-context compaction match long-context agent accuracy with lower real-world serving costs?
NVIDIA-affiliated research authors
This is a first-record evidence baseline. It does not establish independent verification of every research result or completion of future commitments.
Claim and evidence
The November 3 arXiv technical report 'You don't need long context' describes a compaction strategy under which selected frontier open-weight models maintained much of their performance across reported tasks. The authors estimate later tokens can be as much as seven times costlier than earlier tokens under their studied serving assumptions. The experiment includes long-horizon software engineering tasks.
Limitations and uncertainty
The sevenfold figure is an upper-bound cost comparison in a particular compute model, not a universally measured API bill reduction. Task selection, model architecture, inference configuration and compaction strategy may affect conclusions. The report is not an independently replicated cost study.
Explicit acceptance criteria
- Compare identical model-task settings with/without compaction under reproducible workloads.
- Report compute and end-to-end dollars per successful task, not only nominal token cost.
- Rerun across multiple providers, open-weight models and long-horizon evaluation sets.
Dated baseline events
The entries below capture original source and publication milestones. They do not imply that later claims have already been fulfilled or independently replicated.