01 · Foundation
From Prompts to Context
Prompt engineering asked: what words make the model behave? Context engineering asks a bigger question: of everything that could go into the window — system prompt, tool definitions, retrieved documents, conversation history, memory — what is the smallest high-signal set that makes the right outcome likely?
The shift happened because the job changed. A one-shot prompt is a single string you polish. An agent runs for hundreds of turns, generating more candidate context every step — tool results, files, search hits. Someone has to decide what stays in the window. That curation is the discipline.
context-window-anatomy.txt
CONTEXT WINDOW (200k tokens)
==============================================================
system prompt ~2k identity, rules, output format
tool definitions ~4k schemas the model can call
memory / files ~3k persisted facts, project state
retrieved documents ~20k RAG chunks, search results
conversation history ~40k prior turns + tool results
current turn ~1k the actual question
--------------------------------------------------------------
remaining budget ~130k headroom for reasoning + output
Every token above competes for the model's attention.
Curation of this stack IS context engineering.
Mental model
Prompt engineering optimizes a
string. Context engineering optimizes a
state that changes every turn. The prompt is just one resident of the window — and rarely the largest one.
What lives in the window
🧭
System prompt
Identity, rules, output contract. Small but read first — highest leverage per token.
🔧
Tool definitions
Every schema costs tokens on every call. Unused tools are pure attention tax.
📚
Retrievals
RAG chunks, search results, file reads. The biggest and most variable spend.
💬
History
Prior turns and tool results. Grows without bound unless you compact it.
🧠
Memory
Facts that survive across sessions — loaded selectively, not wholesale.
🎯
The actual task
Often the smallest slice of the window. Everything else exists to serve it.
01 · Foundation
The Window as a Resource
Context windows grew from 4k to millions of tokens, and the naive conclusion was that curation stopped mattering. The opposite is true: LLMs have an attention budget, and every token in the window draws it down.
attention-budget.js
// Why bigger windows don't solve the problem
//
// Transformer attention is pairwise: every token attends
// to every other token. n tokens => n^2 relationships.
tokens_in_context = 10_000 // relationships ~ 10^8
tokens_in_context = 100_000 // relationships ~ 10^10
// Training data contains far more short sequences than long
// ones, so the model has less practice at long-range recall.
// Result: an "attention budget" that depletes as the window
// fills — needle-in-haystack scores fall as n grows.
rule_of_thumb = 'find the smallest set of high-signal tokens'
This is architectural, not a bug to be patched. Attention compares every token with every other token, and models see far fewer book-length sequences during training than short ones. So recall quality degrades gradually as the window fills — a model can pass a needle-in-a-haystack benchmark and still lose track of your constraint from forty turns ago.
The one rule
Find the
smallest possible set of high-signal tokens that maximize the likelihood of the outcome you want. Every technique in this guide is a corollary of this rule.
Consequences
Two prompts with identical instructions can perform differently purely because one carries 60k tokens of stale tool output. Token count is a quality knob, not just a cost knob — treat additions to the window as spends that must justify themselves.
01 · Foundation
Context Rot
Long sessions degrade in recognizable ways. Distraction: the model fixates on irrelevant detail carried in the window. Poisoning: a hallucinated or wrong statement enters history and later turns build on it as fact. Drift: instructions from the system prompt slowly lose their grip as thousands of newer tokens pile on top.
session-degradation.log
# Symptoms of context rot in a long agent session
turn 04 ✓ answers cite the right files
turn 12 ✓ still sharp, follows system rules
turn 23 ✗ repeats a step it already completed
turn 31 ✗ confuses config of project A with project B
turn 42 ✗ ignores output-format rule from system prompt
turn 55 ✗ hallucinated tool result treated as fact <- poisoning
# Mitigations
- compact history before symptoms appear (turn ~20, not ~50)
- prune stale tool results; keep decisions, drop transcripts
- re-anchor: restate task + constraints after compaction
- never let an unverified claim become "established fact"
Poisoning is the worst one
Distraction wastes tokens; poisoning corrupts them. Once a false claim is treated as established, every subsequent turn inherits it. The fix is upstream: verify before writing into history, and prune aggressively when you spot contamination.
Rot countermeasures
Compact earlyPrune tool resultsRe-anchor the taskVerify before recordingFresh session per phase
The strongest countermeasure is structural: split long work into phases and give each phase a fresh window seeded with a written handoff — decisions, open questions, file paths — rather than the raw transcript.
02 · Techniques
System Prompt Altitude
System prompts fail at two altitudes. Too low: brittle if-else logic written in prose, which shatters on the first case you didn't anticipate. Too high: "be helpful and accurate" — vague guidance that falsely assumes shared context. The right altitude is heuristics plus success criteria: strong enough to guide behavior, flexible enough to let the model handle novelty.
Wrong altitude
Hardcoded if-else chains in prose
Rules that contradict each other
"Be helpful" with no criteria
Edge cases patched one by one
Right altitude
Principles with reasons attached
Source of truth named, not inlined
Explicit success criteria
Model decides within guardrails
system-prompt-too-low.md
# Too low: brittle if-else logic hardcoded in prose
If the user asks about refunds AND the order is older than
30 days AND the item is not electronics, then say X, unless
they are a premium member, in which case say Y, but if they
already contacted support twice, escalate with template Z...
# Fragile: every new edge case needs another rule,
# and rules start contradicting each other.
system-prompt-right-altitude.md
# Right altitude: heuristics + clear success criteria
You are a support agent for Acme. Resolve issues in as few
turns as possible.
Principles:
- Refunds: policy doc is the source of truth — read it
before quoting terms. Cite the section you used.
- Escalate to a human when the user is frustrated or the
case needs account access you don't have.
- Never invent order details. Look them up or ask.
Output: short paragraphs, no bullet lists in replies.
Test for altitude
Hand your system prompt a case it never mentions. If the model has principles to reason from, you're at the right altitude. If it needs a new rule, you're too low.
02 · Techniques
RAG vs Long Context vs Search
Three strategies compete for how knowledge enters the window: pre-retrieve it (RAG), dump it all in (long context), or let the model hunt for it (agentic search). None dominates — they trade latency, cost, and precision differently.
📦
RAG
Embed, index, retrieve top-k before inference. Fast and cheap at runtime; quality depends entirely on the retriever. Wins for high-volume Q&A over stable corpora.
🐋
Long context
Load the whole document set. Zero retrieval infrastructure, but pays the attention tax on every call. Wins when the corpus is small and cross-references matter.
🔍
Agentic search
The model greps, ranks, and reads iteratively — curating its own window. Slowest, most precise. Wins for codebases and investigative tasks.
agentic-search.ts
// Agentic search: the model curates its own context
async function answer(question: string) {
// 1. cheap wide net — filenames, headers, metadata
const candidates = await grep(keywords(question));
// 2. the model decides what deserves window space
const relevant = await model.rank(candidates, { keep: 5 });
// 3. read ONLY those, then answer
const docs = await Promise.all(relevant.map(readFile));
return model.complete({ question, context: docs });
}
// Trades latency for precision: each step spends tokens
// deciding what NOT to load.
Hybrid is the default in 2026
Production systems layer them: metadata filters narrow the corpus, embeddings rank it, and an agentic loop decides what actually enters the window. The question is not which strategy — it's where each one sits in the funnel.
02 · Techniques
Compaction & Summarization
History grows every turn; the window doesn't. Compaction replaces the oldest span of conversation with a dense summary that preserves decisions, open questions, and references while dropping raw transcripts and dead ends.
compaction.ts
// Rolling compaction: summarize the tail, keep the head sharp
async function compact(history: Turn[]): Promise<Turn[]> {
const KEEP_RECENT = 10;
const old = history.slice(0, -KEEP_RECENT);
const recent = history.slice(-KEEP_RECENT);
const summary = await model.summarize(old, {
keep: ['decisions', 'open questions', 'file paths', 'bugs'],
drop: ['raw tool output', 'dead ends', 'pleasantries'],
maxTokens: 800,
});
return [systemNote(summary), ...recent];
}
// Prune tool results even earlier — a 20k-token page fetch
// is rarely worth 20k tokens after the fact.
const pruned = toolResults.map((r) =>
r.ageInTurns > 3 ? truncate(r, 200) : r,
);
What survives compaction
Decisions + rationaleOpen questionsFile paths / IDsConstraints stated by the userRaw tool output — droppedDead-end explorations — dropped
Compact before rot, not after
Compaction triggered at the window limit inherits fifty turns of drift. Trigger it on a schedule (every N turns) or on a signal (task phase completed) while quality is still high.
02 · Techniques
Memory Architectures
Memory is context that outlives the window. The 2026 consensus pattern is unglamorous and effective: files. A small always-loaded index, one fact per file, grep for recall. Structured enough for the model to maintain, transparent enough for a human to audit.
memory-layout.md
# memory/ — file-based agent memory (persists across sessions)
memory/
MEMORY.md # index — one line per fact, always loaded
user-prefers-bun.md # semantic: stable fact about the user
project-goals.md # semantic: what we're building and why
2026-08-12-debug.md # episodic: what happened, what worked
# MEMORY.md (the only file loaded every session)
- [Prefers Bun](user-prefers-bun.md) — use bun, not npm
- [Project goals](project-goals.md) — neon guide collection
- [Debug session](2026-08-12-debug.md) — types/ dir was stale
# Recall = grep the index, open only what's relevant.
# Write-back = update files when facts change. Delete wrong ones.
Episodic vs semantic
Semantic memories are stable facts — preferences, project goals, architecture decisions. Episodic memories record what happened — a debugging session, a failed approach. Semantic memories get updated in place; episodic ones accumulate and age out. Confusing the two is how memories go stale: yesterday's episode masquerading as a permanent fact.
Memory hygiene
A memory system needs a delete path more than it needs a write path. Wrong memories are worse than no memories — they're poisoning with persistence.
03 · Economics
Prompt Caching
Agents resend nearly identical context every turn. Prompt caching exploits that: mark a breakpoint, and everything before it is processed once and reused at a fraction of the input price — with faster time-to-first-token as a bonus. For a 50-turn session the difference is not marginal; it's the cost model.
request-with-cache-control.json
// Cache breakpoints: everything before the marker is reused
// across calls at ~10% of the input price.
{
"model": "claude-sonnet-5",
"system": [
{
"type": "text",
"text": "<system prompt — stable, never edit per-turn>",
"cache_control": { "type": "ephemeral" }
}
],
"tools": "<tool defs — stable, defined once>",
"messages": [
{ "role": "user", "content": "<history grows append-only>" }
]
}
cache-ordering.txt
# Cache-aware ordering — stable prefix, volatile suffix
[system prompt] never changes ── cached
[tool definitions] never changes ── cached
[memory snapshot] changes rarely ── cached
[history] append-only ── mostly cached
[current turn] changes every call ── paid full price
# Cache killers (invalidate everything after the edit point):
✗ timestamps or request IDs interpolated into the system prompt
✗ reordering tools between calls
✗ editing old messages instead of appending new ones
Design for the cache
Cache-awareness is a context-engineering constraint, not an afterthought: stable content floats to the front, volatile content sinks to the back, and history is append-only. If your architecture edits old messages, you are paying full price for the whole window every call.
03 · Economics
Token Budgets
You cannot engineer what you don't measure. Instrument tasks with a budget: how many input tokens did this answer earn, and where did they go? The usual finding is that tool results dominate spend — fetched pages, file dumps, logs — most of which are never referenced again after the turn that loaded them.
budget-assembly.ts
// Budget-aware context assembly
const BUDGET = 40_000; // input tokens this task has earned
const parts = [
{ name: 'system', tokens: 1_800, required: true },
{ name: 'tools', tokens: 3_200, required: true },
{ name: 'memory-index', tokens: 400, required: true },
{ name: 'retrieved-docs', tokens: 22_000, required: false },
{ name: 'history', tokens: 31_000, required: false },
];
// Spend on required parts first, then highest-signal optional
// parts; truncate or summarize whatever no longer fits.
// Measure per-task: most agents waste 30-60% of input tokens
// on tool results nobody reads twice.
Budget levers
✂️
Truncate tool results
Cap raw results at ingestion; keep a pointer to re-fetch if the full version is needed later.
🪓
Trim the tool belt
Every tool schema rides along on every call. Remove tools the task can't use.
📊
Cited-vs-loaded ratio
Track tokens loaded vs tokens the answer actually used. Low ratios point at over-retrieval.
04 · Execution
Anti-patterns
Most context failures are one of four recurring shapes:
🚰
Kitchen-sink prompt
Every rule ever written, pasted into system. Instructions dilute each other; the important three drown in the trivial forty.
♾️
Unbounded history
No compaction strategy — the session just grows until quality quietly collapses.
📄
Duplicated retrieval
The same document fetched into history three times because nobody checked what was already loaded.
🧟
Stale memory
Facts recorded once, never revisited. The user switched to PostgreSQL in March; the agent still recommends Mongo.
The common root cause
All four are additive bias: it always feels safer to add context than to remove it. The discipline is subtractive — every token must earn its place in the window.
04 · Execution
Context Checklist
Click items to mark complete. Progress tracked per session.
0 / 9 complete
Context Engineering Checklist
System prompt is at the right altitude — heuristics, not if-else proseCritical
Stable prefix ordering: system → tools → memory → historyCritical
Cache breakpoints set on system prompt and tool definitionsCost
No timestamps / request IDs interpolated into cached blocksCost
Compaction triggers before quality degrades, not at the window limitQuality
Tool results pruned or truncated after N turnsQuality
Memory index loaded every session; full memories loaded on demandMemory
Wrong or stale memories get deleted, not appended toMemory
Retrieval is measured: tokens loaded vs tokens actually citedMeasure