It has probably happened to you: at the start of a long conversation with an AI assistant you set a few rules ("always answer concisely", "the client's name is Rossi, not Russo") and after a while the model seems to ignore them. Or you open a new chat and the assistant remembers nothing of what you discussed the day before. This is neither absent-mindedness nor ill will: it depends on how language models handle information, and in particular on the context window.
What the context window is
A language model has no memory in the human sense. Every time it generates an answer, it reads all the text supplied to it at that moment: the system instructions, the previous messages in the conversation, any attached documents and your latest question. This body of text is the context, and the maximum amount the model can consider at once is called the context window.
The window is measured in tokens, the fragments of text that models work on. Sizes vary greatly from model to model: some handle a few thousand tokens, others hundreds of thousands. A large window makes it possible to analyse long documents or extended conversations, but it does not solve every problem.
Why long conversations lose details
When the limit is exceeded
If the conversation grows beyond the window, something has to go. Many services drop the oldest messages or replace them with a summary. Either way, details from the beginning can be lost.
Even within the limit
Even when all the text fits in the window, the model does not pay equal attention to every part. Information in a very long context may be handled less precisely, especially if it is buried among a lot of other material. In addition, instructions given at the start can be "overridden" by later requests that contradict them even partially.
The accumulation effect
Long conversations accumulate attempts, corrections and discarded ideas. The model sees all of it and may confuse an abandoned version with the final one.
How chatbot "memory" works
Some services offer a memory feature that lets the assistant remember information between conversations. Technically the model does not change: it is the application that saves certain information (your preferences, your job, ongoing projects) and feeds it back into the context of subsequent conversations.
This has two practical consequences:
- memory itself takes up space in the context;
- what is stored is a selection, sometimes an inaccurate one, of what you said. It is worth checking periodically what has been saved, correcting errors and deleting anything you do not want remembered.
| Type of "memory" | Where it lives | Duration |
|---|---|---|
| Conversation context | In the text sent to the model | Until the chat ends or the window fills up |
| Custom instructions | Saved by the application | Until you change them |
| Cross-conversation memory | Saved by the application | Until you delete it |
| Model knowledge | In the parameters | Fixed at training time |
How to work better with context
- One conversation per topic. If you change subject completely, open a new chat: you avoid irrelevant information confusing the model.
- Recap the fixed points. In a long conversation, now and then write a summary of the decisions made: "To recap: client Rossi, delivery on the 15th, formal tone".
- Repeat important instructions when they are relevant, instead of trusting the model to remember them from the start.
- Provide only useful material. Attaching ten documents when you need one dilutes attention.
- Say where to look. With long documents, specify the section: "answer based on chapter 3".
- Start afresh when things get tangled. If answers deteriorate after many exchanges, ask for a summary of the situation and paste it into a new conversation.
- Use custom instructions for stable preferences such as language, tone or format, instead of repeating them in every chat.
Context and costs
In pay-per-use services, every message involves sending the entire context to the model. A very long conversation therefore costs more with each new exchange, even if the latest question is short. That is one more reason to keep conversations focused.
Context and confidentiality
Everything in the context, including attached documents and saved memory, is sent to the provider with every request. If you enable cross-conversation memory, information shared for a specific task could reappear in a different context. Think carefully, therefore, about what you let it remember, especially on accounts also used for work.
In short
- The context window is all the text the model considers when it answers.
- Beyond the limit old messages are dropped or summarised; even within it, details can slip.
- Chatbot memory is text saved by the application and fed back into the context.
- Focused conversations, periodic recaps and essential material improve the answers.
- Whatever is in the context is sent to the provider: choose carefully what to let it remember.
