Prompt engineering is over. You're a context engineer now.

A block of dark metal on a workbench, mostly carved away, with heaps of shavings on the floor. What remains at its centre is a small form that glows electric violet.

A 300-token prompt beat a 113,000-token one on the exact same question. Same model, same task, and the short one won. That gap is the whole discipline in one number.

Most teams doing "" are quietly doing the opposite. The reflex is to add: more history, more retrieved documents, more tools, more instructions. It feels like diligence. It is making agents worse, and the measurements are not subtle about it.

The field already settled the easy part. gave way to context engineering, and nobody serious argues otherwise. I called it the new CMS: what your model knows before it writes a word. Anthropic defines it as "curating and maintaining the optimal set of " during inference, where the curation "happens each time we decide what to pass to the model." Cognition calls it "effectively the #1 job of engineers building AI agents." What the field has not internalized is the hard part: the default move, assemble everything relevant into the window, is the antipattern, not the practice.

Everyone agrees context engineering is the job. Almost everyone does it backwards.

Read the definitions closely and they all point at subtraction. Andrej Karpathy's version is "just the right information for the next step." The operative word is just. Not all the relevant information. The right information, and no more.

Then look at how the work actually gets framed. AWS Prescriptive Guidance describes context engineering as "presenting all of the relevant information the needs" and hands teams a checklist: system prompt, plus user profile, plus memory, plus tools, plus retrieval. That checklist is the accumulation reflex wearing a lab coat. Everything on it is relevant. That is exactly the problem. Relevance is not the bar. A model can drown in things that are all individually relevant.

This is the trap I keep watching teams walk into. They frame context work as "what else can we give the model to help it," and in doing so they institutionalize the failure mode. The competitive question for 2026 is not what you can add. It is what you have the discipline to leave out.

A bigger window is not more memory. Long context rots.

The escape hatch everyone reaches for is the itself. Just wait for the million-token model and the problem disappears. Measurement closed that hatch.

Chroma tested 18 models, including , , , and , and found what they named : "performance grows increasingly unreliable as input length grows." This is where that 300-versus-113,000 number comes from. On the benchmark, a focused ~300-token input beat the same content padded to ~113,000 tokens, across every model family. The window was there. The models could not use it.

It gets more counterintuitive. Chroma found models "perform better on shuffled than on logically structured ones." Coherent, well-organized prose hurt retrieval. Even a single sentence measurably dropped performance, and the damage grew with length. Your intuition about what reads well to a human is not a proxy for what a model can attend to.

Anthropic names the mechanism. Models have an "" that "depletes" with each new token, a consequence of transformer attention scaling with the square of the sequence length and of training data where short sequences dominate. Context has "diminishing marginal returns." Databricks saw the same curve from the other side: accuracy started falling around 32,000 tokens, and Gemini 2.5 agents past 100,000 tokens began favoring "repeating actions from history" over forming new plans.

I have made the model-size version of this argument before, that a smaller model with the right context beats a fed a junk drawer. This is the sharper corollary. It is not only that the model is rarely your bottleneck. It is that a fixed model gets measurably dumber as you fill its window, so every "let's wait for longer context" line on your roadmap is a bet against the data.

The failure modes have names. That is what makes this engineering.

Long context does not fail randomly. It fails in specific, reproducible, named ways, and naming them is the difference between a debuggable ticket and a shrug.

Drew Breunig cataloged four. : a lands in the context and then gets referenced again and again as if it were true. : the accumulated context grows large enough to overwhelm what the model learned in training. : superfluous content in the window degrades the output even when it is harmless. : parts of the context contradict each other, and the model has no way to know which to trust.

Each has receipts. On tool confusion, a Llama 3.1 8B failed a benchmark with 46 tools available and succeeded with 19, even though all 46 fit inside the 16k window. The Berkeley Function-Calling Leaderboard shows the general version: every model tested does worse as you hand it more tools. On context clash, a Microsoft and study found a 39% average performance drop when a prompt was "" into fragments that arrived across turns, and OpenAI's fell from 98.1 to 64.1 on the same task when the information came in conflicting pieces instead of one clean block.

The ACE researchers added two more to the vocabulary: "," where optimization collapses your context toward short generic instructions, and "," where a rewrite step nukes hard-won detail. You cannot manage what you cannot name. Teams that adopt this vocabulary triage a regression in an afternoon. Teams that say "the model is being flaky" go buy a bigger one to fix what is actually a curation bug.

The real wins are architectural, not better wording.

Here is the part that reframes the job. The fixes that actually move benchmarks are not cleverer prompt text. They are architecture decisions that keep tokens out of the window in the first place.

Anthropic's own example is blunt. Presenting tools to an agent as on-demand code APIs, instead of loading every tool definition up front, cut one workflow from 150,000 tokens to 2,000. That is a 98.7% reduction from a structural change, not a prompt tweak. Their context editing feature, which auto-clears stale tool results mid-run, combined with a memory tool, improved performance 39% over baseline while using 84% fewer tokens on a 100-turn web-search evaluation. Context editing alone was worth 29%. And it ships as tunable knobs, trigger, keep, clear_at_least, exclude_tools, not a magic button.

Where the leverage actually lives

The highest-ROI context work is in your harness and tool design, not your prompt file. That makes it platform engineering, which is a moat. Prompt text is not.

Isolation is the other big lever, and it is honest about its cost. Anthropic's research system, an Opus 4 lead delegating to subagents that each work in a separate window and report back only a summary, beat single-agent Opus 4 by 90.2% on their evaluation. It also burned up to 15 times the tokens. Powerful, not free. Production teams make the same trade quietly. Ramp's internal coding agent writes roughly 30% of the company's merged pull requests by pre-building environments and spawning isolated child sessions. 's memory system, as reverse-engineered by practitioners, deliberately injects only about 15 lightweight summaries and skips entirely, throwing away most of the raw history on purpose.

Subtraction done wrong is worse than doing nothing, and nobody agrees how to cut.

Now the sharp edge. Everyone ships as the answer, summarize the old context and start fresh with the summary. But compaction is itself a named failure mode, and the industry is openly at war with itself over it.

The ACE paper (arXiv 2510.04618, headed to ICLR 2026) has the smoking gun. A single monolithic rewrite step collapsed an agent's context from 18,282 tokens at 66.7% accuracy down to 122 tokens at 57.1%. That is below the 63.7% you get from doing no adaptation at all. Their fix is the opposite of compaction: "," where you make small incremental "" to a structured context and almost never rewrite it wholesale. That approach gained 10.6% on the benchmark using a smaller open-source model.

What 'just compact it' can cost you

One rewrite step collapsed an agent's context from 18,282 tokens to 122, and its accuracy dropped below doing nothing at all. Auto-compaction ships as a default in most agent stacks today, silently deleting detail you paid for.

And yet auto-compaction is everywhere, shipped by the same companies who know better. Claude Code now fires compaction earlier, around 75% of the window instead of 90%, and treats it as an improvement. ByteDance ships both answers at once: their DeerFlow agent "manages context aggressively, summarizing completed subtasks," while their OpenViking system preserves full detail in a tiered filesystem and loads ~100-token abstracts before pulling the full text. One company, two opposite bets, unresolved.

The most interesting dissent reframes the whole fight. The Distill project argues that "context deduplication is not an optimization, it is a correctness measure," and does it deterministically, no LLM in the loop, 30-40% fewer tokens in about 12 milliseconds, precisely because an LLM rewrite "varies per run." Even compaction's defenders hedge: the practitioner canon says never rely on it for critical rules, move those to a file that survives any compression. As of mid-2026, there is no settled answer on how to cut. The only wrong move is to inherit your vendor's default and never look at it.

The best moves are counterintuitive. Tidying by instinct gets them wrong.

The playbooks that actually work break the naive rules, which is exactly why a team optimizing for "cleaner, shorter context" will do the wrong thing.

Manus, which has published more hard-won detail than almost anyone, keeps failed actions and stack traces in the context on purpose. "Erasing failure removes evidence," they write. "And without evidence, the model can't adapt." Recovering from a visible error is, in their words, one of the clearest signs of real behavior. So the tidy instinct, delete the ugly failed turn, is the one that hurts.

They also treat hit rate as "the single most important metric for a production-stage agent." Cached input tokens run about $0.30 per million versus $3.00 uncached, a 10x swing, so a single unstable token near the start of your prompt, a timestamp, say, can quietly detonate your bill by invalidating the cache for everything after it. That is why their rule is "mask, don't remove" tools: pulling a tool definition out mid-session breaks the cache and confuses the model that saw earlier references to it. Even their good moves have a cost. Their "recitation" trick, rewriting a todo.md to keep the goal in recent attention, ate roughly a third of all actions.

This is context engineering as a boring function boundary, not a vibe. In the Vercel AI SDK, convertToModelMessages silently drops unhandled data parts, and a tool's toModelOutput decides what its result actually contributes to the window. Nobody calls those seams "context engineering." They are exactly where it happens. I have argued that agent memory is mostly hoarding, that the real skill is knowing what to forget. This is the complication to that: sometimes the skill is knowing what ugly, redundant, failed thing to keep.

Prompt engineering was writing. Context engineering is editing. And every editor eventually learns the same lesson the hard way: the strongest thing you can do to a draft is cut it. The teams that win the next couple of years will not be the ones who fit the most into the window. They will be the ones with the nerve to throw most of it out, and the receipts to prove which cut helped.

Glossary

Terms

agentic: Describes AI systems that pursue a goal over many steps on their own, choosing actions and using tools, instead of answering a single prompt and stopping.
AppWorld: A benchmark that tests AI agents on everyday tasks across simulated apps such as email, shopping and music, where the agent must write code that calls the apps' APIs correctly.
attention budget: The idea that a model can only attend well to so much text at once, and every extra token uses some of that capacity. It is why a bloated prompt can make a model worse.
brevity bias: A failure where automatic prompt rewriting keeps shrinking instructions toward short, generic text, dropping the specific details that made them useful.
compaction: Summarizing a long agent conversation into a shorter version so work can continue within the model's limit. It saves space but can silently drop details.
context clash: A failure where new information in a model's context contradicts what is already there, so the model gets confused and answers worse.
context collapse: A failure where rewriting a long working context into a new version wipes out accumulated detail, leaving a thin summary the model can no longer rely on.
context confusion: A failure where superfluous material in a model's context, such as too many tools or documents, leads it to use the wrong thing or answer worse.
context distraction: A failure where a very long context makes a model lean on its accumulated history instead of what it learned in training, reducing the quality of its answers.
context engineering: Deciding what information goes into a model's working memory at each step: instructions, tools, documents and history. Good practice often means leaving things out.
context poisoning: A failure where a mistake or hallucination gets into a model's context and is then treated as fact, so later steps keep building on the error.
context rot: The drop in a model's accuracy as its input grows longer, even when the task stays the same. More text in the prompt does not mean better answers.
context window: The amount of text a language model can read and consider at one time, including the prompt, tools and conversation so far. Anything beyond it is not seen by the model.
delta updates: Making small, incremental edits to a structured body of notes or instructions instead of rewriting it whole, so earlier detail is preserved.
distractor: Irrelevant text placed in a model's input to see whether it gets sidetracked. Even one distractor can lower accuracy, and the effect grows as inputs get longer.
Frontier model: One of the most capable AI models available at a given moment, from the leading labs. What counts as frontier keeps moving as newer models arrive.
Grow-and-refine: A way to improve an AI agent's context by making small incremental edits to a structured notebook of lessons, instead of rewriting or compressing it each time, which loses detail.
Hallucination: When an AI model states something false or invented with full confidence, such as a fake citation or a feature that does not exist. It matters because the output reads as fluent and plausible.
KV-cache: Saved intermediate results of a language model's attention over the text so far, reused instead of recomputed. A stable prompt prefix hits the cache, making calls faster and cheaper.
LLM: A large language model: a neural network trained on huge amounts of text to predict and generate language. It powers chatbots and coding assistants.
LongMemEval: A benchmark that tests whether chat assistants can recall and reason over facts buried in long conversation histories spanning many sessions.
Multi-agent: A setup where several AI agents, often a lead plus helpers, split a task and report back. It can cover more ground at the cost of more compute and coordination.
Needle in a haystack: A test that hides a single fact (the needle) inside a long stretch of unrelated text (the haystack) and asks the model to find it. It probes how well a model uses long context.
Prompt engineering: Crafting the wording and examples in an instruction to a language model so it gives better answers. It covers one prompt, while context engineering covers everything the model is shown.
Quantization: Storing a model's numbers at lower precision so it runs cheaper and faster, at the price of some accuracy. Hosted models can be served this way, which may quietly change their quality.
Retrieval-augmented generation: A setup where the system first looks up relevant documents and puts them in the prompt, so the model answers from fresh, specific sources instead of memory alone.
Sharded prompt: A task split into fragments that arrive over several turns of a chat instead of all at once. Studies show models do noticeably worse on instructions delivered this way.
Token (LLM): The chunk of text, often a word or part of a word, that a language model reads and writes. Context limits and pricing are counted in tokens, so they measure how much text a model handles.

Tools

ChatGPT: OpenAI's conversational AI assistant, available as a web app and mobile apps.
Claude Opus 4: A model in Anthropic's Claude Opus line, the most capable tier of the Claude 4 generation.
Claude Sonnet 4: A model in Anthropic's Claude Sonnet line, the mid-sized tier of the Claude 4 generation that balances capability, speed and cost.
Gemini 2.5: Google's Gemini 2.5 model family, with Pro, Flash and Flash-Lite variants, available through the Gemini API.
GPT-4.1: An OpenAI language model without a reasoning step, built for following instructions and calling tools, with a very long context window.
Llama 3.1: Meta's open-weight language model family, released in several sizes. Developers can fine-tune, distill and deploy it themselves.
o3: An OpenAI reasoning model for complex tasks involving analysis across text, code and images, including math, science and coding.
Qwen3: A series of large language models from the Qwen team at Alibaba Cloud, with both dense and mixture-of-experts variants.
Salesforce: A customer relationship management platform that unifies company data, customer touchpoints and employee tools.