Editing Openai/6939b23e-87cc-8001-bdd9-9546d1143ce9 (section)

=== Because the model does NOT store the tokens inside its memory. ===

Instead:
* The context window is stored in your computer's RAM, not inside the model weights.
* The model re-reads all 130,000 tokens every time it generates a new token.

This is the key idea:

The model does not “store” the context — it ''recomputes'' it for each token.

===== - Model weights (e.g., 3GB): static, fixed parameters =====
* Token embeddings + attention cache (e.g., up to ~130k tokens): stored externally, passed into the model every cycle

So the model’s size is independent of how much conversation history it can process.

It’s like a calculator:
The calculator doesn’t grow when you feed it more input numbers.
You just give it bigger stacks of numbers.