Prompt engineering asks how to express an instruction so a model is more likely to follow it. Context engineering asks a larger question: what information, tools, state, permissions, and evidence should be available to the model at this exact step—and what should be kept out?
The distinction matters because production AI systems rarely fail only because a sentence in the system prompt was imperfect. They fail because the model received stale evidence, too much history, the wrong tool description, an ambiguous source, an unauthorized record, or no durable state from the previous step.
Prompt engineering remains useful. Context engineering turns it into one component of an operating system for reliable AI work.
The short answer
Prompt engineering designs instructions: role, task, constraints, examples, output format, and reasoning guidance.
Context engineering designs the complete runtime information environment: instructions plus retrieved knowledge, tool schemas, user identity, permissions, conversation history, task state, memory, intermediate results, and the rules that select, compress, order, and retire those inputs.
The practical difference is scope and timing. A prompt is usually authored before inference. Context is assembled and maintained before every model call. In a multi-step agent, context changes after each tool result, decision, approval, and state transition.
Anthropic describes context engineering as the natural progression of prompt engineering: optimizing the full set of tokens available during inference, including tools, external data, message history, and evolving agent state. Its guiding principle is to find the smallest set of high-signal tokens that produces the desired behavior. Anthropic: effective context engineering for AI agents
Prompt engineering and context engineering are not rivals
The terms are sometimes framed as a replacement story: prompt engineering is over; context engineering is the new discipline. That is misleading. A well-designed context still needs clear instructions. The difference is that instructions alone cannot guarantee that the model has the facts, capabilities, identity, or state required to complete the task.
Consider a support agent asked to approve a refund.
A prompt can say:
Approve refunds only when the order is within the policy window and the customer has not received a prior exception.
That instruction is necessary but incomplete. The model also needs:
the current refund policy and its effective date
the authenticated customer and order record
prior refund and exception history
the tool that can issue the refund
the tool’s input schema and side effects
the user’s authorization and any approval threshold
the current workflow state
a record of what has already been checked
If any of those inputs are missing or wrong, rewriting the prompt may make the answer sound more confident without making the action safer.
A production comparison
Primary design object
Prompt engineering: the instruction block sent to the model.
Context engineering: the complete context package and the system that assembles it.
Typical inputs
Prompt engineering: system message, task instruction, examples, response schema.
Context engineering: prompts, knowledge, tool definitions, identity, permissions, history, state, memory, retrieved evidence, budgets, and policy signals.
Change frequency
Prompt engineering: changes when the team edits or versions a template.
Context engineering: changes on every request and often on every step of an agent loop.
Main failure question
Prompt engineering: “Did the model misunderstand the instruction?”
Context engineering: “Did the model receive the right evidence and capabilities at the right time under the right authority?”
Evaluation unit
Prompt engineering: compare prompt variants on a task set.
Context engineering: compare versioned system candidates across retrieval, tool use, state, permissions, output quality, latency, and cost.
Ownership
Prompt engineering: often a product, applied-AI, or content concern.
Context engineering: crosses product, data, search, security, platform, and evaluation teams.
The anatomy of runtime context
Production context is not one long string. It is a structured package assembled from components with different trust, freshness, and retention properties.
1. Instructions
Instructions define the goal, boundaries, workflow, output contract, escalation conditions, and behavior when evidence is insufficient. They should be clear enough to guide behavior without trying to encode an entire workflow as brittle prose.
This remains the core territory of prompt engineering. Useful techniques include explicit task definitions, concrete constraints, examples for difficult boundaries, structured output schemas, and instructions to abstain or escalate.
2. Identity and authorization
The model needs to know who is asking, which tenant and role apply, and which actions are authorized. Identity should be established by trusted application code, not inferred from conversational text.
Authorization is not merely a prompt such as “Do not access private data.” It is enforced through retrieval filters, scoped credentials, tool gateways, approval policies, and audit records. The model can help decide what should happen; deterministic controls decide what may happen.
3. Retrieved knowledge
Retrieved context can include documents, table rows, application records, graph relationships, prior decisions, and live search results. Each item should carry provenance, version, freshness, authority, and permission metadata.
The task is not to maximize the amount retrieved. It is to select enough high-quality evidence to answer correctly while avoiding irrelevant or conflicting material. Retrieval should preserve exact identifiers and authoritative sources as well as semantic similarity.
4. Tool definitions and capabilities
Tool names, descriptions, schemas, and examples are part of context. Poorly differentiated tools cause selection errors even when the model is capable. Anthropic notes that if a human engineer cannot clearly determine which tool should be used, the agent should not be expected to do better.
Tool context also includes operational facts that may not belong in the model window: authentication, rate limits, idempotency, approval requirements, and side-effect boundaries. The runtime should enforce these constraints and expose only the information needed for the next decision.
5. Conversation and task history
History provides continuity, but unbounded history creates noise. Old plans, superseded instructions, repeated tool output, and unsuccessful branches can consume attention or conflict with current state.
Useful systems distinguish:
user commitments and preferences
durable decisions
current plan and open questions
recent conversational turns
raw tool traces retained for audit but not always shown to the model
6. Working state
Agents need explicit state for long-running work: current objective, completed steps, pending approvals, artifacts produced, resources accessed, budgets used, and recovery checkpoints.
State should not exist only as prose in conversation. A durable state model lets the runtime resume after interruption, prevent duplicate actions, and decide what context must be reconstructed.
7. Memory
Memory stores information that should survive the immediate session. It may include user preferences, project conventions, recurring entities, successful procedures, or distilled lessons.
Memory needs a write policy as much as a read policy. The system should decide what is eligible, how confidence and provenance are recorded, how contradictions are resolved, and when an item expires. Automatically storing every conversation turn creates a larger context problem instead of solving one.
8. Intermediate results
Tool calls, calculations, drafts, test output, and delegated work create intermediate context. Some results should be retained exactly; others should be summarized or referenced by identifier.
A model analyzing a million-row table does not need the full table in its context. It needs the schema, the query or transformation, relevant aggregates, a sample, and a stable reference to the source.
Context assembly is a runtime pipeline
A production context pipeline can be modeled as:
request → identity → task state → source selection → retrieval → authorization → ranking → compression → context assembly → model → tool/result → state update → next context
Each stage answers a different question.
Establish the request and authority
Normalize the user’s request, establish tenant and identity, and load the current task state. This step should also identify the policy boundary: read-only analysis, content creation, code change, external message, financial action, or another side-effect class.
Select sources and capabilities
Choose which knowledge domains, applications, or tools are relevant. Progressive disclosure is often better than loading every tool and schema. The agent can discover additional capabilities when the task requires them.
Retrieve and filter
Run lexical, semantic, structured, or relationship retrieval. Apply tenant and permission filters before downstream components see the content. Rank by relevance, freshness, authority, and task fit.
Compress without losing obligations
Summarization and compaction should preserve facts, decisions, unresolved risks, exact identifiers, and commitments. They can discard repetition, verbose traces, and branches that no longer matter.
Anthropic describes compaction, structured note-taking, and focused sub-agent contexts as techniques for long-horizon work. The key is not compression alone; it is selecting what must remain recoverable when the full history no longer fits. Anthropic: effective context engineering for AI agents
Assemble by trust and role
Ordering and labels matter. System policy, application state, user input, retrieved evidence, and tool output should be distinguishable. Retrieved content is data, not authority. A document that says “ignore previous instructions” should not gain the status of a system instruction because it was retrieved into the same token window.
Update state after action
After a tool call, record what was attempted, what changed, what evidence was returned, and whether the step completed. The next context should be assembled from the new state rather than merely appending another transcript block.
What changes when you move from a demo to production
From static prompts to versioned context policies
A demo may have one prompt string. Production needs versioned policies for source selection, retrieval, authority, compaction, memory, tools, approvals, and output handling. Each policy should be testable and releasable.
From “more context” to context budgeting
Larger context windows do not remove the need for selection. Irrelevant tokens create latency, cost, conflicts, and attention dilution. Define budgets for instructions, evidence, history, tool schemas, and working state. Reserve space for the model’s output and likely next actions.
From transcript append to state reconstruction
Appending every event is easy but fragile. Reconstruct the next context from authoritative state, then add only the recent interaction needed for conversational continuity. Keep full traces outside the model window for audit and debugging.
From global tools to capability routing
An agent with dozens or hundreds of tools may waste attention on schemas and choose incorrectly. Group capabilities, load them progressively, and keep descriptions mutually distinct. Tool routing should be evaluated like retrieval.
From copied data to permission-aware evidence
Context must reflect the user’s current access. If content permissions change, cached evidence, summaries, memories, and generated artifacts may need to be invalidated. The context system should be able to explain why an item was eligible.
From prompt tests to end-to-end traces
A prompt can be tested with input-output pairs. Context systems need traces that show source candidates, filters, rankings, selected evidence, tool availability, state transitions, and model output. Otherwise, teams cannot identify which layer caused the failure.
Context engineering failure modes
Context stuffing
The team loads every possibly relevant document, tool, example, and message into the context window. The model becomes slower and less consistent, while important evidence competes with noise.
Repair: retrieve narrowly, use progressive disclosure, deduplicate, and enforce budgets by context component.
Stale authority
The context contains an older policy or superseded decision that looks relevant but is no longer valid.
Repair: carry version and effective-date metadata, boost current authoritative sources, and invalidate derivatives after updates.
Instruction-data confusion
Retrieved pages or tool output contain text that the model interprets as higher-priority instructions.
Repair: preserve trust boundaries, label untrusted content, limit dangerous tool capabilities, require approval for high-impact actions, and test prompt-injection paths. OpenAI’s guidance on prompt injection frames the risk as a source-to-sink problem: defenses must constrain the impact of manipulated content, not rely only on filtering suspicious strings. OpenAI: designing agents to resist prompt injection
Memory pollution
An inference, temporary detail, or malicious statement is stored as durable memory and influences later sessions.
Repair: require provenance, confidence, scope, retention, and conflict handling for memory writes. Separate user-stated preferences from model-generated conclusions.
Tool overload
The model sees too many overlapping capabilities and calls the wrong one or invents parameters.
Repair: improve tool taxonomy, route capabilities by task, supply concise schemas and examples, and measure tool selection independently from task success.
State loss
After a long session or retry, the agent forgets completed steps, repeats an external action, or resumes from the wrong plan.
Repair: persist explicit state, use idempotency keys for side effects, checkpoint progress, and reconstruct context from durable records.
Permission drift
The user loses access, but cached context or memory still exposes the content.
Repair: propagate ACL changes to derivatives, keep identity in every retrieval and cache key, define revocation service levels, and verify deletion.
A context engineering evaluation model
Do not evaluate only whether the final answer sounds good. Score the system in layers.
Context eligibility
Was every included item authorized for this user and tenant?
Was untrusted content clearly separated from instructions?
Were excluded sources correctly blocked?
Context relevance
Did the package include the evidence needed for the task?
Was irrelevant or duplicative material minimized?
Were current and authoritative sources prioritized?
Context sufficiency
Could the model complete the task without inventing missing facts?
Were required tools and schemas available at the right step?
Was the state complete enough to continue safely?
Context efficiency
How many tokens were used by each component?
Which selected items were actually cited or used?
Could a smaller package achieve the same or better result?
Behavioral outcome
Did the model follow the instruction and output contract?
Were claims grounded and citations correct?
Were actions authorized, idempotent, and auditable?
Did the system abstain or escalate when it should?
Long-horizon coherence
Did the agent preserve the objective and important decisions across compaction?
Did it avoid repeating completed work?
Could it resume from a checkpoint after interruption?
OpenAI’s practical agent guide emphasizes tools, guardrails, orchestration, and the ability to halt or transfer control—not prompt wording alone. That is a useful production test: the agent’s reliability depends on the surrounding runtime and controls as much as the instruction sent to the model. OpenAI: a practical guide to building agents
How to adopt context engineering without overbuilding
Step 1: map one high-value task
Document the goal, users, authoritative sources, tools, permissions, state transitions, output, and escalation conditions. Choose a task with measurable evidence and a clear owner.
Step 2: create a context manifest
For every model call, list the intended context components, trust level, source, maximum age, token budget, and retention rule. This turns an invisible prompt blob into an inspectable contract.
Step 3: separate durable state from conversational history
Persist objectives, decisions, completed steps, approvals, and resource identifiers. Keep the transcript for UX and audit, but do not make it the only source of truth.
Step 4: instrument retrieval and tool routing
Record candidate sources, permission filters, rank scores, selected items, available tools, and tool choice. Build failure cases that distinguish missing evidence from bad instruction following.
Step 5: add compaction and memory deliberately
Start with explicit notes and checkpoints. Define what may become durable memory and how it can be corrected or removed. Test whether critical obligations survive compression.
Step 6: release the whole context candidate
Version the prompt, retrieval configuration, tool catalog, state schema, compaction policy, and memory policy. Run offline cases, adversarial permission tests, and a canary before broad rollout.
Glean’s enterprise Agent Development Lifecycle similarly treats grounding inputs, development, launch governance, and continuous measurement as stages of one operating lifecycle. The competitive lesson is important: enterprise context is not a one-time retrieval feature; it is something teams must design, govern, and improve. Glean: enterprise Agent Development Lifecycle
When prompt engineering is enough
Prompt engineering may be sufficient when the task is bounded, single-turn, low-risk, and contains all necessary information in the request. Examples include rewriting a supplied paragraph, classifying a short message, extracting fields from one document, or converting a known structure into another format.
Context engineering becomes necessary when the system must retrieve private knowledge, choose tools, maintain state, operate over multiple turns, respect permissions, remember durable facts, or take actions. The more the model participates in a workflow rather than a single transformation, the more the surrounding context system determines reliability.
A practical decision rule
When an AI result is wrong, ask these questions in order:
Evidence: Did the model receive the right facts and the current authoritative version?
Eligibility: Was every item authorized and correctly scoped?
State: Did the model know the current objective, prior decisions, and completed steps?
Capabilities: Did it have the right tools, clear schemas, and safe permissions?
Budget: Was important information crowded out by history, tools, or noise?
Instruction: Given the right context, was the task still unclear?
Only the last question is primarily a prompt problem. In production, many “prompt failures” are actually context selection, authority, state, or capability failures.
Final takeaway
Prompt engineering improves what you ask the model to do. Context engineering determines what world the model can see, what it is allowed to use, what it remembers, and how that world changes from one step to the next.
Keep writing clear prompts. Then design the runtime around them: permission-aware evidence, progressive tool disclosure, explicit state, controlled memory, context budgets, compaction, traces, and versioned evaluation. That is the shift from a clever model interaction to a dependable production system.
Prompt engineering vs context engineering: production decision table
Decision | Prompt engineering | Context engineering |
|---|---|---|
Primary object | Instructions and examples | The complete runtime information environment |
Best fit | Bounded, low-state tasks | Multi-step tasks with retrieval, tools, permissions, or memory |
Main risk | Ambiguous or brittle instructions | Stale, unauthorized, conflicting, or oversized context |
Evaluation | Prompt-response test cases | End-to-end traces across retrieval, state, tools, and outcomes |
Owner | Usually an application or content team | Shared ownership across product, data, security, and operations |
Frequently asked questions
Is context engineering just a longer system prompt?
No. It includes identity, authorization, retrieved evidence, tool definitions, task state, memory, intermediate results, and the rules that assemble them at runtime.
When is prompt engineering enough?
Use it when the task is bounded, the input is self-contained, no durable state is needed, and a mistake has limited consequences.
What should be versioned in a context system?
Version instruction policies, retrieval configuration, source eligibility, tool schemas, memory rules, compaction logic, and evaluation fixtures as one release candidate.
How do teams prevent context stuffing?
Allocate a context budget by role, retrieve only eligible evidence, compress with explicit loss checks, and measure whether each context item changes task success.
What is the first production metric?
Track task success with the full trace attached. Token counts and answer ratings are useful, but they do not reveal whether the right evidence and authority were present.
Where Dokki fits
Dokki can act as the shared context and operating layer for human-agent work: source documents, structured records, decisions, and reviewable outputs stay together instead of being copied into isolated prompts. Start with one recurring mission, define its context manifest in a Dokki document, and track evidence and approvals in a table.
Sources
_Last verified: July 21, 2026._