Your agent has been working on a problem for thirty steps. It has read six documents, called nine tools, and — somewhere around step nineteen — quietly forgot what it was originally asked to do. Not because the model is weak, but because everything an agent "knows" has to fit in one prompt, and that prompt has a ceiling. This lesson is about spending those tokens well. The industry now calls it context engineering, and it's become the highest-leverage skill in agent work.
There is no memory — only what you resend
Start from the uncomfortable fact: the model remembers nothing between calls. Every "memory" feature in every agent framework is your code deciding what to put back into the next prompt. An agent that recalls the goal does so because the goal was resent. One that recalls a customer's name does so because something stored it and looked it up.
Once you internalize that, memory design becomes a budget problem: a fixed number of tokens per step, and a growing pile of candidates competing for them.
Short-term memory — the running context
Short-term memory is whatever your code assembles for this step: the goal, the conversation so far, the agent's own thoughts, and the tool results it has collected. In LangGraph terms, it's the state (lesson 5).
The problem is arithmetic. Each step appends a thought, a tool call and a result — and the whole thing is resent every step. A thirty-step run doesn't send thirty small prompts; it sends thirty increasingly enormous ones. Cost grows quadratically with run length, and eventually the window simply overflows.
Long-term memory — RAG, pointed at yourself
Long-term memory means things that survive across runs and sessions: user preferences, past decisions, learned facts, prior cases. It can't live in the context window, so it lives outside — usually a database or vector store — and is retrieved back in when relevant.
Read that again and you'll recognize it: long-term agent memory is literally RAG applied to the agent's own history — same machinery from the RAG course, different corpus. Interviewers notice when a candidate spots that reuse.
def build_context(user_id, goal, recent_steps):
facts = memory_store.search(query=goal, user_id=user_id, k=3)
return [
system_prompt,
f"What we know about this user:\n{format(facts)}", # long-term
f"Goal: {goal}", # never dropped
*recent_steps, # short-term
]Crucially, you store extracted facts, not transcripts("prefers Hindi", "owns model MX-200", "case #4471 open"). Months of raw conversation fits nowhere and costs a fortune to re-send.
Wait — my agent is at step 30 and the context is full. Now what?
You compact: summarize the middle, keep the ends.
def compact(state, budget_tokens=8000):
keep = [state["goal"]] # NEVER summarize the goal
recent = state["messages"][-6:] # last steps verbatim
older = state["messages"][:-6]
if not older:
return keep + recent
summary = llm_summarize(
older,
instruction="Summarize what was tried, what was learned, and what "
"remains unknown. Preserve exact IDs and numbers.",
)
return keep + [summary] + recentResult
goal (verbatim) + summary of steps 1–24 + steps 25–30 (verbatim)
Bounded context regardless of run length — the goal and the freshest observations survive intact.
Three rules are encoded there, and each exists because of a real failure: the goal is never compressed (a summarized goal drifts, and the agent wanders), recent observations stay exact (they drive the next decision), and the summary instruction explicitly preserves ids and numbers — the details summaries otherwise destroy, leaving an agent that knows it found an order but not which one.
Context engineering — the real skill
Prompting is writing good instructions. Context engineering is deciding what deserves the limited tokens at each step across everything competing for them:
| Candidate | Include when… |
|---|---|
| System instructions & tool schemas | always — but keep them tight; they're paid for every step |
| The goal | always, verbatim |
| Recent observations | always — they drive the next decision |
| Older trajectory | summarized, not verbatim |
| Retrieved documents | only the few chunks this step actually needs |
| Long-term memories | only those relevant to the current goal |
The same model with a curated context outperforms it with a bloated one — and costs less. That sentence is the whole discipline, and it's why "context engineering" started appearing in job descriptions.
Context poisoning — errors that compound
A failure mode unique to long agent runs. At step 3 a tool returns something wrong, or the model hallucinates a detail into its thought. Every subsequent step reads that as established fact. By step 20 the agent is confidently building on a fiction — and errors compound rather than average out.
Mitigations: verify important tool results rather than trusting them; have the agent mark uncertainty explicitly ("unconfirmed") instead of stating it flatly; re-ground from the original source rather than from your own earlier summary; and cap run length so a poisoned run ends rather than sprawling.
Selection-round radar: "How does an agent remember things?" expects: nothing is innate → short-term = the context window → long-term = external store retrieved back in → compaction when it grows → the goal is never summarized. Adding "long-term memory is RAG over the agent's own history" is the line that makes an interviewer nod.
Memory and privacy — the part people forget
The moment an agent stores things about users, you've built a personal-data system. Store extracted facts, not transcripts; keep retention bounded; let users see and correct what's stored (memories go stale and wrong); scope retrieval strictly to that user; and keep a hard never-store list — card numbers, passwords, OTPs, unmasked government identifiers.
In Indian interviews, naming the DPDP Act and "PII masked before storage and before any third-party call" signals you've thought past the demo.
Common mistakes
- Believing the model remembers anything between calls.
- Letting the trajectory grow unbounded until the window overflows mid-run.
- Summarizing the goal along with everything else — the agent then drifts.
- Summaries that drop ids and numbers, leaving the agent unable to act.
- Storing raw conversation transcripts as "long-term memory".
- No way for users to review or correct stored facts.
Quick recap
| Concept | One-liner |
|---|---|
| Reality | no innate memory — only what your code resends |
| Short-term | the running context: goal, thoughts, tool results |
| Long-term | external store of extracted facts, retrieved when relevant (RAG on itself) |
| Compaction | summarize the middle; keep goal and recent steps verbatim; preserve ids |
| Context engineering | budgeting limited tokens across instructions, memory, retrieval and history |
| Poisoning | an early error becomes fact for every later step — verify and cap runs |
| Privacy | facts not transcripts, bounded retention, user-correctable, hard never-store list |
Practice Zone — PYQs from real selection rounds
Six MCQs and two tasks — write the compaction function, then design a support agent's memory including what must never be stored.
Where does an agent's short-term memory actually live?
Asked in
Long-term agent memory is usually implemented as:
Asked in
An agent's trajectory has grown to 40 steps and is approaching the context limit. What's the standard fix?
Asked in
Context engineering is best described as:
Asked in
What is 'context poisoning' in a long agent run?
Asked in
Which memory design is most appropriate for a personal assistant used daily for months?
Asked in
Hands-on tasks:
Write the logic that keeps an agent's context under a token budget: what do you keep verbatim, what do you summarize, and what do you drop?
Asked in
A support agent talks to the same customers repeatedly. Design its memory: what's stored, where, for how long, and what must NOT be stored.
Asked in
FAQ
Do bigger context windows remove the need for memory design?
No — they raise the ceiling and change nothing else. You still pay per token every step, attention still degrades when relevant details are buried, and cross-session memory still has to live outside the window. Bigger windows buy headroom, not a strategy.
How does an agent decide what's worth remembering long-term?
Usually an explicit extraction step: after a run, a cheap model pulls out durable facts ("prefers email over phone") and discards the rest. Some systems let the agent write memories deliberately via a tool — which is cleaner, because you can log and review what it chose to store.
Is compaction the same as summarizing the conversation?
It's summarization with rules: which parts stay verbatim (goal, recent steps), which get compressed (older trajectory), and what must survive compression (ids, numbers, unresolved questions). Naive summarization of everything is how agents lose the thread.
Next lesson: when one agent isn't enough — and when it is — Lesson 7: Multi-Agent Systems →