This article distills the technical substance from Chilia's Chinese essay and the linked reference material, while removing the source post's cover art, promotional cards, community QR code, and other non-technical wrapper content.

Original Chinese discussion: Zhihu
Source post: Qing Ke Ai WeChat article

Context Engineering: The Runtime Layer Prompt Engineering Misses

Prompt engineering is useful when the job is a single interaction. Agent systems are different. Once a model starts calling tools, reading files, parsing logs, and revising plans over many turns, the real bottleneck becomes the evolving context state.

That is where context engineering comes in. It is the practice of deciding what stays in the context window, what gets pushed out of it, what can be retrieved later, what should be summarized, and what must remain stable for caching to work.

This shift matters because strong agents do not fail only from bad reasoning. They also fail because their working memory becomes too long, too noisy, too expensive, or too unstable.

TL;DR

• Context engineering is the runtime discipline of managing an agent's changing short-term memory.
• The main enemy is not only context length but context rot: more tokens, more noise, weaker recall, and slower inference.
• The safest first move is usually reversible offload: move bulky outputs to files or external storage, then fetch them only when needed.
• Summarization is useful, but it is lossy and should usually come after reversible compaction is exhausted.
• Sub-agent isolation lets long or noisy work happen in a separate context window.
• KV cache stability depends on append-only histories and deterministic request prefixes.

If you want broader architecture context, continue with A Visual Guide to LLM Agents: Types, Architecture & How They Work, PlugMem: Better AI Agent Memory at Lower Context Cost, the Token Calculator, and the Agent Technology Hub.

1. Why Agent Context Breaks Down

Every tool call adds new observations to the running conversation. In production, those observations are often much longer than the model's final answer:

• shell logs,
• file reads,
• JSON payloads,
• web-page extracts,
• MCP responses,
• intermediate plans and todos.

Over time, the agent's chat history stops behaving like a clean reasoning trace and starts behaving like a noisy execution log. Even when a model technically supports a long context window, practical quality usually drops earlier. Recall becomes less reliable, latency rises, and useful signals get buried under stale or redundant details.

That is the operational meaning of context rot. The goal of context engineering is therefore not "fit everything." It is to keep only the smallest high-signal state needed for the next correct step.

2. Strategy One: Offload and Retrieve Instead of Hoarding

The cleanest way to control context growth is to stop treating the context window as the only memory surface.

One strong pattern, popularized in recent agent discussions around systems like Manus and Claude Code, is to treat the file system as an external memory layer. Large artifacts move out of the active prompt and into durable storage, while the prompt keeps only lightweight pointers such as file paths, URLs, or short notes.

That makes the compression reversible:

• a web page can be replaced by its URL,
• a long file read can be replaced by a path,
• a huge command output can be written to disk and referenced later.

This is better than naive truncation. Truncation destroys information immediately. Reversible offload preserves the option to recover details only when they become relevant.

Just-in-Time Retrieval vs. Pre-Inference Retrieval

This same philosophy also changes how retrieval works. Instead of relying only on pre-indexed RAG pipelines, many modern agent stacks increasingly favor just-in-time retrieval:

• search when needed,
• search with tools the model already understands,
• search incrementally based on the last result.

For example, an agent can rg for a symbol, inspect a file, then narrow the search again. That often matches how humans work in a codebase or document corpus: not by preloading everything, but by iteratively querying the environment.

This does not make RAG useless. It does mean RAG is no longer the only default. In practice, simpler retrieval paths are often more robust for fast-moving agent tasks.

3. Strategy Two: Summarize Only When Reversible Compression Runs Out

Sometimes the context window is still close to full even after aggressive offload. At that point the system needs a lossy compression step, which is where summarization belongs.

Summarization is powerful because it frees space quickly, but it has an obvious cost: the model loses direct access to exact prior details. That is why it should be treated as a fallback rather than the default first move.

A safer summarization pattern looks like this:

1. Dump the full conversation history to durable storage.
2. Replace the long history in-context with a structured summary.
3. Keep the most recent full tool interactions if possible.
4. Allow the agent to re-read the archived transcript when exact details matter again.

This turns a lossy summary into a recoverable summary. The prompt becomes shorter, but the original evidence is still reachable.

Claude Code's /compact flow is a good example of this design philosophy. The model continues with a summary of the earlier session while the exact transcript remains available in a backing file for later lookup.

4. Strategy Three: Use Sub-Agents as Context Isolation Boundaries

Another way to reduce context contamination is not to compress more aggressively, but to split the work.

When a task contains several distinct subtasks, a main agent can delegate them to specialized sub-agents. The key benefit is not only specialization. It is context isolation.

Each sub-agent gets its own context window, its own instructions, and often its own tool permissions. That means a verbose background investigation does not have to flood the main thread with every intermediate read and log line. The main agent only needs the returned summary or artifact.

This provides several practical wins:

• the main agent stays focused on the top-level objective,
• large exploratory traces remain isolated,
• permissions can be narrowed for safer delegation,
• cheaper models can be used for simpler read-only work.

It is also a useful way to reason about multi-agent architecture itself. A sub-agent is not only a worker. It is a boundary that prevents one part of the task from polluting another part's active memory.

5. Strategy Four: Design for KV Cache Stability

Prompt caching matters because agent workloads often repeat a large stable prefix across many short generations. If the reusable prefix can be cached, the system can reduce both latency and repeated compute.

But prompt caching is fragile. It works only when the prefix remains stable. Change whitespace, reorder serialized keys, inject a timestamp near the top, or rewrite earlier messages, and the cache can miss.

That pushes agent design toward a few concrete rules:

• prefer append-only histories over editing old content,
• keep system prompts stable,
• serialize JSON deterministically,
• isolate dynamic metadata from the stable prefix whenever possible.

A minimal cache-control block looks like this:

{
  "cache_control": {
    "type": "ephemeral",
    "ttl": "1h"
  }
}

And a cache-aware API request often keeps the stable instructions at the top while appending only new user or tool state below:

curl https://api.anthropic.com/v1/messages \
  -H "content-type: application/json" \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -d '{
    "model": "claude-opus-4-6",
    "max_tokens": 1024,
    "cache_control": {"type": "ephemeral"},
    "system": "You are a helpful assistant that remembers our conversation.",
    "messages": [
      {"role": "user", "content": "My name is Alex. I work on machine learning."},
      {"role": "assistant", "content": "Nice to meet you, Alex! How can I help with your ML work today?"},
      {"role": "user", "content": "What did I say I work on?"}
    ]
  }'

The deeper point is architectural, not API-specific: a cache-aware agent should treat its prefix as an asset worth protecting.

6. How These Pieces Fit Together

The most useful takeaway from context engineering is that there is no single magic trick. Strong agent systems usually combine several layers:

• offload bulky data,
• retrieve details just in time,
• summarize only when necessary,
• isolate noisy work behind sub-agents,
• keep stable prefixes cacheable.

That stack is more realistic than betting everything on a larger context window. Bigger windows help, but they do not solve noise management, retrieval strategy, cache behavior, or orchestration hygiene by themselves.

This is also why context engineering sits naturally beside memory systems like PlugMem. Memory decides what knowledge should persist across tasks. Context engineering decides what evidence should be active right now inside a limited runtime budget.

Conclusion

Context engineering is best understood as the operations layer of agent intelligence. It decides how an agent stays coherent when tasks get longer, tools get noisier, and intermediate state grows faster than the prompt window can handle.

The practical lesson is simple: do not treat the context window as a dumping ground. Treat it as scarce working memory. Offload what can be recovered, summarize only when forced, isolate work that should not leak back, and keep reusable prefixes stable enough to cache.

That is what turns a capable model into a durable agent.

References

1. Anthropic. Effective Context Engineering for AI Agents
2. minusx.ai. Decoding Claude Code
3. Manus. Context Engineering for AI Agents: Lessons from Building Manus
4. Anthropic. Platform Documentation
5. Chilia. Original Chinese essay
6. Qing Ke Ai. WeChat source post