Context length is how much text a model can use for a request, measured in tokens. Your instructions, earlier messages, supplied documents, retrieved passages, tool output, and the answer all need room. A model's published maximum and the context selected in your app are different numbers; available memory can constrain the usable setting further.
This guide is useful when a model forgets a detail, rejects a long prompt, slows down as a conversation grows, or fails after you increase context. If a short prompt is also wrong, begin with answer-quality troubleshooting rather than assuming context is the cause.
Keep three limits separate
| Limit | Where to look | What it establishes |
|---|---|---|
| Published model limit | The exact checkpoint's model card | The context configuration described by the publisher, sometimes with extra requirements |
| Selected model-runner context | App load settings or the running server | The allocation used by this setup |
| Practical memory capacity | A memory estimate and the actual running computer | Whether the selected workload leaves enough RAM or VRAM |
For example, the original Qwen3 4B card describes 32,768 native tokens and a larger limit with YaRN, a context-extension method. Setting any app to a larger number does not establish that the required extension is configured or that your computer can run it.
Estimate the text before raising the setting
Use the token counter with the material you intend to send. Its result is exact only for the tokenizer it actually uses; another model may split the same text differently. Page counts and word counts are weaker proxies, especially for code, tables, or languages other than English.
Suppose a selected context holds 8192 tokens. If instructions and existing messages use about 1500, and you reserve about 1000 for an answer, roughly 5692 remain for new text. That is a planning example, not a measured allocation in your app. Chat templates and tools may add further tokens.
For document chat, find out whether the app supplies the whole file or selected passages. A 200-page PDF can produce a short prompt if search retrieves a few passages; that does not mean the model considered every page.
Start with a modest setting and inspect it
In an Ollama conversation, an explicit starting setting is:
/set parameter num_ctx 4096
Send another short prompt, then inspect ollama ps in a second terminal. Its context and processor information helps distinguish the requested allocation and CPU/GPU placement. Ollama's context documentation describes hardware-dependent defaults; do not assume one default applies to every installation.
In LM Studio, choose the context in the model's load settings and reload it if the app requires that change. Its command-line load tool also supports --context-length. Check the displayed memory estimate before increasing the value. A model already loaded with one allocation may need a new load to use another.
Increase context only when the task needs it and the RAM and VRAM calculator leaves suitable space. Keep the exact model and quantization unchanged during that comparison so you can identify which setting caused the difference.
Why longer context uses more memory
The model keeps conversation memory, often called the KV cache, while processing text. Its size depends on the model architecture, selected context, cache precision, and sometimes parallel requests. Model-weight compression and KV-cache compression are separate settings; choosing a Q4 model does not mean its conversation memory is automatically four-bit too.
Longer prompts also require more processing before an answer starts. If the model fits with a short chat but fails with a long one, reduce context or the material sent before changing drivers or redownloading weights.
Check that the longer setting helps the task
Use harmless material containing a fact near the beginning and another near the end. Ask for both, then inspect the answer against the source. Repeat in a fresh conversation at the setting you intend to use. You want correct answers and acceptable memory use, not simply an app accepting the larger number.
This is a practical check for that material, not proof of perfect long-document understanding. Models can overlook details inside an accepted context. Search can miss a passage, and summaries can drop a qualification. For consequential document work, check the underlying source.
If the required material will not fit, choose a model and machine that can handle it, or use a document-search workflow that still preserves the relationships the question requires. Splitting a comparison across isolated fragments can change the task and hide contradictions.