Memory determines which local models are practical, but “16GB” can mean system RAM, dedicated GPU VRAM, or Apple unified memory. Those are not interchangeable. Identify the memory pool your runtime will use, estimate the model weights plus overhead and context, then leave enough headroom for the operating system and normal work.
Practical starting tiers
| Hardware memory | First text-model class | Practical use |
|---|---|---|
| 8GB total memory | 1B–4B, 4-bit | Learning, short chat, rewriting, light summaries |
| 16GB total or unified memory | 7B–8B, 4-bit | Everyday chat, writing, focused coding, small document sets |
| 24GB total or unified memory | 8B–14B, 4-bit | Stronger general use and moderate document work |
| 32GB total or unified memory | 14B first; larger after testing | More capable assistants and heavier multitasking |
| 8GB dedicated VRAM | 7B–8B, 4-bit | Responsive GPU-local chat with moderate context |
| 12GB dedicated VRAM | 7B–14B, 4-bit | Larger text models, subject to context and backend overhead |
| 16GB dedicated VRAM | 14B, 4-bit | Stronger local text workloads and larger context than 8GB cards |
| 24GB dedicated VRAM | 14B–32B, 4-bit | High-end desktop inference, depending on active parameters and context |
| 64GB+ unified or system memory | 14B–32B first | Larger local experiments and substantial document workflows |
Use these as starting points. The exact result changes with model architecture, quantization, context length, runtime, GPU offload, multimodal components, and what else is running.
The four memory costs that matter
1. Model weights
The model file is the largest obvious cost. A 4-bit quantization uses roughly half a byte per parameter before format and metadata overhead. That means a dense 8B model often lands around the 4GB–6GB range, while a dense 32B model can require roughly four times as much.
Do not apply the parameter count blindly to mixture-of-experts models. Total parameters affect storage, while active parameters can affect compute per token. The exact memory behavior depends on how the model and runtime load the experts.
2. Runtime and backend overhead
The runtime needs memory for buffers, kernels, model metadata, tokenization, and other working state. Loading the exact model file into a memory pool with no remaining headroom is therefore not a viable plan.
3. Context and key-value cache
The context window stores the conversation, retrieved passages, system instructions, and generated tokens. Longer context normally increases memory use. A model that works at a modest context can fail or slow sharply when the context target grows.
Start with the runtime’s ordinary default or a deliberately modest target. Increase context only after confirming that the model, speed, and memory pressure remain acceptable.
4. The rest of the computer
The operating system, browser, editor, document index, chat history, and every other process need memory too. Apple unified memory makes this competition especially visible because the CPU and GPU draw from the same pool.
RAM, VRAM, and unified memory
System RAM
System RAM is the main pool for CPU inference and for any model portions that do not fit in dedicated GPU memory. It also holds the operating system and ordinary applications. A CPU-only setup can sometimes load a model that is too large for a GPU, but output may be too slow for comfortable daily use.
Dedicated GPU VRAM
Dedicated VRAM is a separate high-bandwidth pool attached to a discrete GPU. It is usually the key constraint for fast local inference on NVIDIA or AMD hardware. A model that fits mostly or entirely in dedicated VRAM generally performs better than one that repeatedly moves data through system memory.
Windows may report “shared GPU memory.” That is system RAM available to the GPU, not additional dedicated VRAM. Do not add it to the dedicated-VRAM number and assume the result behaves like a larger graphics card.
Apple unified memory
Apple Silicon combines CPU and GPU access to one memory pool. That avoids a small fixed VRAM partition, but it does not give the model exclusive use of all installed memory. The operating system, applications, model, context, and graphics all compete for the same pool.
A 16GB Apple Silicon Mac is a sensible 7B–8B machine, not a 16GB discrete-GPU equivalent with separate system RAM. A 32GB or 64GB Mac gives substantially more room, but the same headroom rule still applies.
A practical planning formula
For a first-pass text-model estimate, think in this order:
- Start with the actual quantized model file size. Use the specific artifact, not only the family’s parameter count.
- Add runtime overhead. Use the RAM/VRAM calculator rather than assuming the file size is the total.
- Add context overhead. Larger context, batch sizes, and multimodal inputs can materially change memory demand.
- Reserve operating-system and application headroom. A desktop machine still needs to function while the model is running.
- Reject a plan that depends on constant swapping. Technical launch is not the same thing as a usable workflow.
Worked examples
8GB Apple Silicon Mac
Start with 1B–4B 4-bit text models. Use short context and close memory-heavy applications. This tier is useful for learning and light tasks, but it is not a comfortable large-model or large-document machine.
16GB Apple Silicon Mac
Start with a 7B–8B 4-bit model. Leave room for macOS and the applications beside the runtime. A 14B model may be possible in some configurations, but it should not be the default recommendation for a beginner who wants a responsive general-purpose setup.
Windows PC with 16GB RAM and no discrete GPU
Use a small model and expect CPU-limited speed. Adding more system RAM can help a larger file load, but it does not create the throughput of a supported discrete GPU. Choose this path for privacy experiments, short tasks, or learning—not for pretending an office laptop is a high-end inference workstation.
Windows PC with 32GB RAM and 12GB dedicated VRAM
Start with 7B–14B 4-bit models. Plan primarily around 12GB of dedicated VRAM for fast inference, while using the 32GB system RAM for the operating system, applications, and controlled offload. Watch whether longer context forces meaningful spill into system memory.
64GB Apple Silicon Mac
Start with a 14B–32B 4-bit model based on the job. Higher memory makes larger models practical, but speed and context still decide whether they are useful. Compare output on your real task before keeping the larger model as the default.
Multimodal and document workflows need extra headroom
Image-capable models may load projectors, vision encoders, or other components in addition to the text model. Document chat adds extraction, embeddings, indexes, retrieved passages, and larger prompts. Image generation can have very different memory behavior from text generation.
Do not use a text-only memory table as a guarantee for these workflows. Start with one small representative input and observe the complete pipeline.
What success looks like
A memory plan is successful when:
- the model loads without exhausting the intended memory pool;
- a normal prompt returns a response at a usable pace;
- context can grow to the intended working size without crashes;
- the operating system and essential applications remain responsive;
- document or multimodal components can run beside the model;
- the setup does not depend on unexplained background swapping.
Common mistakes
Planning from parameter count alone
Parameter count is useful, but the exact artifact, quantization, architecture, context, and runtime determine real memory demand.
Treating 16GB RAM as 16GB VRAM
They are different pools with different bandwidth and different competition from the rest of the system.
Treating unified memory as entirely available to the model
macOS and every other application use the same pool. Reserve headroom.
Maximizing context before the first successful run
Long context can create a memory problem that looks like a model-fit problem. Start modestly, then increase.
Calling a model practical because it launches once
A useful setup must survive normal prompts, context growth, and the applications you need beside it.
Next step
Use What local LLM can my computer run? for a direct hardware-first path, or enter the exact file and context in the RAM/VRAM calculator.