What is Context Engineering?

Updated July 2026

Context engineering is the discipline of assembling everything a large language model sees at inference time, the system prompt, retrieved documents, tool outputs, conversation history, and working memory, so it has exactly the information it needs to complete a task well. It's the practice that replaced prompt engineering as the core AI-building skill through 2025 and 2026.

For business leaders, context engineering means the difference between an AI system that guesses and one that knows. A support agent that pulls the customer's order history, refund policy, and prior tickets into its working context before answering will get the answer right more often than one running on a clever prompt alone. The prompt is one input among many; context engineering is the job of managing all of them together, on every single call, not just the first one.

Why Context Engineering Replaced Prompt Engineering

Prompt engineering treated getting good AI output as a wording problem: find the magic phrase, the right persona, the perfect instruction structure, and the model would perform. That framing made sense when a single well-crafted prompt was most of what a user controlled. It stopped making sense once teams started building AI agents and multi-step systems where the prompt is a small, fixed part of a much larger, constantly changing pool of information the model has to work from.

The term "context engineering" was popularized in mid-2025 by two prominent voices in AI. Andrej Karpathy called it "the delicate art and science of filling the context window with just the right information for the next step," and argued that "context engineering" describes what production AI teams actually spend their time on far better than "prompt engineering" ever did. Around the same time, Shopify CEO Tobi Lutke described it publicly as "the art of providing all the context for the task to be plausibly solvable by the LLM."

Anthropic's engineering team framed the shift this way in its own guidance for building AI agents: context engineering is "the set of strategies for curating and maintaining the optimal set of tokens" during inference, covering the system prompt, tools, examples, message history, and retrieved data together, not the prompt in isolation. The underlying insight driving the shift: frontier models are strong enough for most business tasks now. The gap between a system that works and one that doesn't usually comes down to what information the model was given, not which words were used to ask for it.

Key Components of Context Engineering

Every context-engineered system is built from the same handful of parts, assembled and re-assembled on every model call.

• System prompt: The standing instructions that define the AI's role, tone, and boundaries. This is what most people mean by "prompt engineering," now just one input among several.
• Memory: Short-term working memory for the current task, plus longer-term memory for facts, preferences, and outcomes that should persist across sessions. See AI memory for how this differs from a model's training data.
• Retrieval-augmented generation (RAG): The mechanism for pulling relevant documents, records, or knowledge base entries into context at the moment they're needed, instead of hoping the model already knows them. Covered in depth in Retrieval-Augmented Generation.
• Tools: The functions, APIs, and systems the model can call, and just as important, the outputs those calls return, which then become part of the context for the next step. See tool use.
• Examples: A small number of well-chosen, representative examples (few-shot demonstrations) that show the model what good output looks like, curated for relevance rather than exhaustive coverage.
• State: The running conversation history, prior actions taken, and intermediate results that let a multi-step agent stay coherent from the first step to the last, rather than treating each turn as a fresh start.

Getting any one of these wrong (a bloated tool list, a stale memory, an irrelevant retrieved document) degrades output quality just as much as a badly worded prompt used to.

How Context Engineering Works in Agentic Systems

In an agentic AI system, context isn't assembled once and left alone. It's rebuilt or updated on every step of the loop: the system gathers what it currently knows (system prompt, memory, retrieved data), the model reasons over that context and decides on an action, tool use executes that action against a real system, and the result gets folded back into context for the next decision. Multi-step agentic workflows can run this loop dozens or hundreds of times before a task is done, which means small context mistakes early on can compound by the end.

LangChain's engineering team has organized the practical work of context engineering into four recurring strategies: write (give the agent a place to save information outside the main context, like scratchpads or notes), select (pull in only what's relevant to the current step, not every document available), compress (summarize or trim context once it grows long), and isolate (split a complex task across multiple agents or sub-contexts so no single context gets overloaded). Most production context engineering work is some combination of these four moves, repeated continuously as the task runs. AI orchestration is the layer that typically manages this on top of a single agent or across several.

Context Window Management and Failure Modes

Every model has a finite context window, the maximum number of tokens it can process at once. But context engineering exists because window overflow isn't the main problem in practice. Research from Chroma tested 18 frontier models, including GPT-4.1, Claude Opus 4, Gemini 2.5, and Qwen3, on inputs ranging from 25 to 10,000 words and found accuracy on simple retrieval and recall tasks degrading steadily as input length grew, well before any model hit its stated token limit. That gradual decline has a name: context rot, the tendency for a model's ability to accurately use information to drop as the amount of context it's holding increases, even on tasks it would ace with a shorter input.

Context rot isn't one failure, it's a cluster of related ones. Writer Drew Breunig catalogued four specific patterns that explain most of what practitioners run into:

• Context poisoning: A hallucination or error gets written into the context (a wrong fact in a memory note, a false belief in a tracked goal) and then gets referenced repeatedly, compounding the mistake instead of correcting it.
• Context distraction: The model leans too heavily on accumulated history and starts repeating past actions instead of reasoning fresh about what to do next.
• Context confusion: Irrelevant information sitting in the context gets pulled into the response, lowering answer quality even when the correct information was also present.
• Context clash: Information added at different points in a long session starts contradicting itself, and the model has no reliable way to know which version is current.

A well-documented real-world example of context poisoning: Google DeepMind's own Gemini Plays Pokemon project, a livestreamed experiment having Gemini 2.5 Pro play through Pokemon Blue autonomously. At one point the agent hallucinated that it needed to find an item called "TEA" (which doesn't exist in that version of the game), wrote that false goal into its persistent context, and then spent hours pursuing an impossible objective before the team could get it unstuck. The fix in production systems isn't a bigger context window, it's actively managing what stays in context and for how long.

Context Engineering vs Prompt Engineering

Prompt engineering didn't disappear, it got absorbed. Writing a clear system prompt is still one piece of the work; it's just no longer the whole job.

Real Use Cases of Context Engineering

Customer support agents: Before answering, the system retrieves the customer's account status, order history, and the specific policy document relevant to their question, then keeps that context available for the rest of the conversation instead of re-fetching it every turn.

Coding agents: Tools like Claude Code and similar coding assistants manage repository context directly, deciding which files, prior edits, and test results stay in the model's working context as a task progresses across many steps, since including the whole codebase at once would blow past any window and invite context rot.

Sales and RevOps agents: An agent qualifying or following up on a lead pulls relevant CRM records, past interactions, and account details through RAG rather than relying on the model's general knowledge of the company.

Long-running autonomous agents: Browser-use and research agents that operate across many steps or sessions depend on deliberate memory management, saving what matters to persistent storage and pruning what doesn't, so the working context stays focused instead of accumulating everything the agent has ever seen.

Key Facts

• Anthropic defines context engineering as "the set of strategies for curating and maintaining the optimal set of tokens" available to a model during inference, spanning the system prompt, tools, examples, and message history, not the prompt alone. Anthropic Engineering
• Andrej Karpathy's June 2025 post describing context engineering as "the delicate art and science of filling the context window with just the right information for the next step" is widely credited with popularizing the term. X / Karpathy
• Chroma's Context Rot research tested 18 frontier models, including GPT-4.1, Claude Opus 4, Gemini 2.5, and Qwen3, across input lengths from 25 to 10,000 words and found accuracy degrading consistently as inputs grew longer, well before hitting any model's stated context limit. Chroma Research
• LangChain's 2026 State of AI Agents survey found 57% of respondent teams now have agents running in production, with output quality, including managing context at scale, cited by 32% as the top barrier to reliable production performance. LangChain
• LangChain's engineering team frames practical context engineering work around four recurring strategies: write, select, compress, and isolate. LangChain Blog
• Google DeepMind's Gemini Plays Pokemon project produced a documented case of context poisoning: a hallucinated in-game goal entered the agent's persistent context and derailed its play for hours before the team could correct it. TechCrunch
• Writer Drew Breunig catalogued four distinct context failure modes, poisoning, distraction, confusion, and clash, that together explain most of what practitioners call context rot. dbreunig.com

Related AI Concepts

• What is Agentic AI? - The autonomous systems that depend on well-engineered context to work reliably
• Context Window - The token limit that context engineering has to work within
• Prompt Engineering - The narrower discipline that context engineering absorbed and expanded on
• Large Language Models - The systems that consume the context assembled for them
• Retrieval-Augmented Generation - The main technique for pulling external knowledge into context
• AI Memory - Short and long-term memory as a context engineering component
• Tool Use - How tool outputs become part of an agent's ongoing context
• Model Context Protocol - The open standard for feeding external tools and data into a model's context consistently
• Agentic Workflows - Multi-step processes where context has to stay coherent across many steps
• AI Orchestration - The coordination layer that manages context across multiple agents or steps

External Resources

• Anthropic: Effective Context Engineering for AI Agents - Anthropic's own guidance on curating context for agent reliability
• LangChain: Context Engineering for Agents - The write, select, compress, isolate framework
• LangChain: State of AI Agents 2026 - Survey data on production agent adoption and quality barriers
• Chroma Research: Context Rot - Benchmark data on performance degradation across 18 frontier models
• Drew Breunig: How Long Contexts Fail - The four named context failure modes
• TechCrunch: Google's Gemini Panicked While Playing Pokemon - A documented real-world case of context poisoning

Part of the AI Terms Collection. Last updated: 2026-07-16