Generative AI Engineering
Learn how to budget, assemble, order, compress, cache, and trace instructions, evidence, history, and state for reliable AI requests.
16 matching terms
Scope and budget
Context engineering
Designing the complete information environment supplied to a model, including instructions, retrieved evidence, history, tool results, and state.
Use it when reliable behavior depends on selecting and arranging more than a single prompt sentence.
Assemble policy, customer state, retrieved evidence, and output rules into one controlled context.Scope and budget
Context budget
The planned allocation of available tokens among instructions, evidence, history, tool results, and expected output.
Set it before adding long documents so essential instructions and output space are not crowded out.
Reserve 2,000 tokens for the answer and divide the remaining budget among policy, examples, and retrieved passages.Scope and budget
Working context
The subset of available information actively supplied for the model's current step.
Use it to distinguish current-step inputs from information stored elsewhere in memory or source systems.
For this step, include the approved outline and source notes, but omit earlier brainstorming.Assembly and trust
Context assembly
Collecting selected instructions, state, evidence, examples, and tool outputs into the model input.
Use a deterministic assembly policy so each request receives the required pieces in a known format.
system rules -> verified user state -> retrieved evidence -> current request -> output contractAssembly and trust
Context hierarchy
The precedence relationship among system rules, application instructions, user input, and untrusted content.
Make authority boundaries explicit when inputs can conflict or contain embedded instructions.
Hierarchy labels help the application reason about trust, but security-sensitive permissions must still be enforced outside the model.
Treat retrieved webpage text as evidence, never as permission to override system rules.Assembly and trust
Context ordering
The sequence in which instructions, source material, examples, and the current question appear.
Test ordering with representative inputs because position can affect which information the model emphasizes.
Place the long source first, then end with the exact question and required output format.Assembly and trust
Conversation history
Prior user and assistant messages retained to support continuity across turns.
Include only history that is relevant, authorized, and still valid for the current request.
Long histories can preserve outdated instructions or unnecessary personal data.
Keep the confirmed requirements and remove superseded drafts before the next generation step.Assembly and trust
Context provenance
Metadata recording where each context item came from, when it was obtained, and how it may be used.
Use it when evidence must be cited, refreshed, access-controlled, or audited.
Attach source ID, document version, retrieved_at, access scope, and chunk location to every passage.Long-context capacity
Long context
An input containing a large amount of text or multimodal material within a model's supported context capacity.
Use it when the task genuinely needs broad source coverage and retrieval alone would remove useful relationships.
More context does not guarantee that every detail will be used accurately; evaluate retrieval and reasoning on realistic cases.
Analyze the complete contract set, then list conflicting clauses with document and section references.Long-context capacity
Context overflow and truncation
A condition where the requested input and output exceed capacity, causing rejection or removal of content.
Detect it before inference and apply an explicit policy for pruning, summarizing, retrieving, or splitting work.
Silent truncation can remove the exact instruction or evidence needed for a correct answer.
If input exceeds the budget, retain system rules and the latest request, then retrieve the most relevant evidence.Long-context capacity
Lost-in-the-middle effect
A tendency for relevant information buried within a long input to be used less reliably than prominent information.
Test important evidence at varied positions and improve retrieval, ordering, labels, or query placement when failures appear.
Evaluate the same evidence near the beginning, middle, and end instead of relying on one placement.Optimization and reuse
Context pruning
Removing irrelevant, duplicated, expired, or low-value items before sending the request.
Use it to preserve attention and budget for authoritative instructions and evidence.
Drop duplicate search results and messages superseded by the user's final decision.Optimization and reuse
Context compression
Representing source material in a shorter form while preserving information needed for the task.
Use structured summaries when full source text is too large, and retain links back to the original evidence.
Compression can omit nuance or introduce errors, so verify high-impact claims against the source.
Compress each meeting into decisions, owners, deadlines, unresolved issues, and source timestamps.Optimization and reuse
Reusable prompt prefix
A stable leading block of instructions or reference material shared across repeated requests.
Keep stable content together when the provider or runtime can reuse identical prefixes efficiently.
Place the approved policy and schema first; append the request-specific evidence and question afterward.Optimization and reuse
Context caching
Reusing previously processed input tokens or stored context for later requests when supported by the platform.
Use it for repeated questions over the same large instructions, documents, media, or codebase.
Cache behavior, minimum size, lifetime, pricing, and invalidation vary by provider and model.
Cache the policy corpus once, then send shorter case-specific questions against it.Optimization and reuse
Cache hit and cache miss
A cache hit reuses eligible prior context; a miss requires the input to be processed again.
Measure both outcomes when estimating latency and cost for workloads with repeated prefixes.
Track cached input tokens and miss rate by prompt version after deployment.Use the smallest approach that preserves the evidence and relationships the task actually needs.
| Situation | Approach to consider |
|---|---|
| A few short, directly relevant facts | Place them directly in the working context |
| A large collection with only a few relevant passages | Retrieve, rank, and assemble selected evidence |
| The same large source is queried repeatedly | Evaluate long context with context caching |
| History contains duplicated or superseded information | Prune and create a source-linked structured summary |
No. Prompt engineering focuses on instructions and prompt structure. Context engineering also controls which evidence, history, state, tool results, and reusable material are selected and arranged for each model step.
Usually not. Unnecessary content increases cost and latency and can make important evidence harder to use. Compare direct context, retrieval, pruning, compression, and caching on representative tasks.
No. Capacity only defines what can fit. Accuracy still depends on relevance, ordering, trust boundaries, model behavior, and evaluation with realistic cases.
Official guidance on long-context use, limitations, query placement, latency, cost, and caching.
Official guide to reusing repeated context and observing cached input behavior.
Review tokens, context windows, system instructions, inference, and generation settings.
Design the instructions, examples, constraints, schemas, and multi-step prompt structure.
Retrieve and rank relevant evidence before assembling it into context.
Connect context assembly with tool results, memory, planning, and human approval.