Quick diagnosis
An out-of-memory failure usually means the complete workload exceeds the usable memory pool, not merely that the model file is large. Count model weights, runtime overhead, context, multimodal components, document processing, and every other application that shares the same RAM, dedicated GPU memory, or Apple unified memory.
The first useful fix is usually to reduce model size or context. Reinstalling the app does not create more memory.
Identify the failing memory pool
Dedicated GPU memory
On a Windows or Linux machine with a discrete GPU, the runtime may fail because the model and working data do not fit in dedicated GPU memory. Some runtimes can offload part of the workload to system RAM, but that can make generation much slower.
Check the actual dedicated-VRAM number. Do not add Windows “shared GPU memory” and treat it as more dedicated VRAM.
Apple unified memory
On Apple Silicon, the model shares one pool with macOS, context, document indexes, the browser, and every other process. A model that appears to fit the installed memory can still fail when the operating system and normal applications consume the remaining headroom.
System RAM
CPU-only inference and partial GPU offload use system RAM. A machine can also have enough total RAM to load a model but too little remaining memory to keep the operating system responsive or grow the context.
First five fixes
- Close memory-heavy applications. Browsers, editors, virtual machines, games, and document-processing tools can consume the same pool.
- Reduce context length. Test a short prompt at a modest context before changing anything else.
- Use a smaller model. Move down a model class rather than forcing the current model to launch through constant swapping.
- Use a more compact model build. A common 4-bit build is normally the practical starting point for local text models.
- Restart the runtime after a failed load. A failed process can leave memory pressure or another model resident.
Diagnose by symptom
| Symptom | Most likely explanation | First action |
|---|---|---|
| The model never finishes loading | Model weights and loading overhead exceed the usable pool | Try a substantially smaller model |
| A short prompt works but long chats fail | Context and key-value cache consume the remaining headroom | Reduce context and start a fresh chat |
| Text chat works but image input fails | Vision components or projectors add memory demand | Test text-only, then a small image with the required components |
| Chat works but PDF import causes failure | Extraction, embeddings, indexes, and retrieved text compete with the model | Test one small document and reduce model size |
| The machine becomes unresponsive | The operating system is swapping heavily | Stop the process and choose a smaller workload |
| A GPU model spills into system RAM | Dedicated GPU memory is exhausted | Reduce model size, context, or GPU-resident layers |
Check the exact model artifact
Model-family names are not enough. Record:
- the exact file or model tag;
- quantization or format;
- total and active parameters where relevant;
- context target;
- runtime and version;
- GPU offload or backend settings;
- whether a vision projector or embedding model is also loaded.
Two downloads with the same family name can have materially different memory requirements.
Context can be the hidden cause
Context holds system instructions, chat history, retrieved passages, attached documents, and generated tokens. A model can load cleanly and then fail only after a long conversation or large retrieval result.
Test with a new chat and a deliberately modest context. If that works, increase context gradually until you find the practical limit for the real workflow.
Document and multimodal workflows
A PDF workflow can run several memory-consuming stages beside the chat model: parsing or OCR, chunking, an embedding model, a vector index, retrieval, reranking, and a longer final prompt. Image-capable models may load a vision encoder or projector in addition to the language model.
Do not diagnose these workflows from the model-file size alone. Run one small representative document or image and observe the complete pipeline.
What success looks like
The fix is successful when the model loads without sustained swapping, returns a normal response, survives the intended context length, and leaves enough memory for the operating system and the tools required beside it.
A model that launches once while making the computer unusable is not a successful configuration.
When to stop and switch
Use a smaller model or different machine when:
- the workload depends on sustained swapping;
- the runtime repeatedly terminates as context grows;
- the document or multimodal pipeline cannot coexist with the model;
- the GPU path repeatedly spills into system RAM and becomes unusably slow;
- closing normal work applications is the only way to keep the model alive.
Next step
Use the RAM and VRAM requirements guide to understand the memory pools, then enter the exact model file and context in the RAM/VRAM calculator.