Context window vs memory: why AI forgets your conversation
By Unlearnt
A context window is the amount of information a language model can work with in one request. Saved memory is information an app keeps for later and may supply to the model again. Neither is the same as the model's trained knowledge. Separating these explains how a chatbot can remember your writing style yet lose track of a deadline.
Context, saved memory and training do different jobs
The model generates its next response from its trained parameters and the context supplied for that request. Context can include instructions, earlier messages, documents and tool results. The window is measured in tokens, which can be words, parts of words or other units. What fits depends on the model and the product using it.
An app can store a note outside that window, then retrieve it into a later request. Saving and retrieving the note does not require retraining the model. A chat history visible on your screen is also not proof that the whole history was supplied to the model for this particular answer.
| Layer | What it contains | What it does not guarantee |
|---|---|---|
| Trained parameters | Patterns learned during training. | That a fact you just typed has become permanent knowledge. |
| Current context | Information supplied for this response. | That every included detail will be used correctly. |
| Saved memory | Information stored outside the request and retrieved when needed. | That every chat detail is saved, retrieved or up to date. |
Worked example: the missing project deadline
Imagine a fictional assistant called Cedar. You tell it, ‘The poster is due on Friday; keep the copy brief.’ Cedar includes that message in its next request and correctly repeats Friday. That is compatible with using context; it does not demonstrate a training update.
Suppose Cedar saves only ‘prefers brief copy’ in a separate note. You open a new conversation. In this example, the app supplies that note but no previous messages. It can tailor the length, but has no supplied evidence for the poster's deadline. A confident date would not make that evidence appear.
If you then attach a project note stating the deadline, Cedar can include it in context and answer from it. This is an illustrative design, not a claim about the memory settings of a particular chatbot.
| Moment | Information supplied | What is supported |
|---|---|---|
| First conversation | Your message with Friday and brief copy. | The deadline and style preference. |
| Fresh conversation | Only the saved brief-copy preference. | The style preference, not the deadline. |
| Project note attached | A note stating the deadline, plus the preference. | Both, subject to reading and using them correctly. |
Why can it forget within the same chat?
As a conversation grows, its messages compete for limited space with other instructions, documents and tool results. Depending on the system, old material may be omitted, selected for retrieval or compressed into a summary. A summary can retain the broad project goal while losing a particular exception or date.
There is also a different failure: a detail can be present but used poorly. Research on long contexts found that performance varied with where relevant information appeared, including weaker performance for information in the middle on the tasks tested. A larger window is more capacity, not a guarantee of perfect recall. One missed answer alone cannot tell you whether a provider reduced its window.
Misconception: remembering means it learned permanently
Repeating your deadline could come from the current conversation, a retrieved note or another supplied document. You cannot infer a change to model weights from that response alone. Likewise, starting a fresh chat does not universally erase or preserve everything: the surrounding app decides which stored information to supply.
For a deadline that matters, provide a short current project note and check the answer against it. If the date changed, make the new date and the superseded date explicit. This gives the system better evidence; it does not make every answer reliable.
Check your understanding
Optional: apply the idea to a new scenario.
An assistant remembers your preference for short replies in a new chat but gives the old date for a meeting you rescheduled yesterday. Does this prove its context window is too small? Give two possible explanations and say what information you would check or supply next.
Answer with the tutor →Free to start, no account needed.