For a first local text model, 16GB of system or unified memory is a useful starting point for a small 4-bit model; an 8GB machine calls for a smaller 1B–4B model and shorter conversations. Memory determines which local models are practical, but “16GB” can mean system RAM, dedicated GPU VRAM, or Apple unified memory. Those are not interchangeable.
Quick checklist for setting up a local LLM:
- Identify which memory pool your runtime will use (system RAM, dedicated GPU VRAM, or unified memory).
- Check the file size of the exact model you want to run (not just parameter count).
- Estimate the runtime overhead and the additional memory needed for context and buffers.
- Make sure to leave enough headroom for your operating system and the applications you will use alongside the model.
- Choose the model size that matches what you have available after these calculations, starting small and scaling up only when you confirm performance is acceptable.
Practical starting tiers
| Hardware memory | First text-model class | Practical use |
|---|---|---|
| 8GB total memory | 1B–4B, 4-bit | Learning, short chat, rewriting, light summaries |
| 16GB total or unified memory | 7B–8B, 4-bit | Everyday chat, writing, focused coding, small document sets |
| 24GB total or unified memory | 8B–14B, 4-bit | Stronger general use and moderate document work |
| 32GB total or unified memory | 14B first; larger after testing | More capable assistants and heavier multitasking |
| 8GB dedicated VRAM | 7B–8B, 4-bit | Responsive GPU-local chat with moderate context |
| 12GB dedicated VRAM | 7B–14B, 4-bit | Larger text models, subject to context and backend overhead |
| 16GB dedicated VRAM | 14B, 4-bit | Stronger local text workloads and larger context than 8GB cards |
| 24GB dedicated VRAM | 14B–32B, 4-bit | High-end desktop inference, depending on the full artifact, runtime, and context |
| 64GB+ unified or system memory | 14B–32B first | Larger local experiments and substantial document workflows |
Use these as starting points. The exact result changes with model architecture, quantization, context length, runtime, GPU offload, multimodal components, and what else is running. Edge-case scenarios such as multi-GPU setups, hybrid CPU/GPU arrangements, or other unusual configurations can introduce additional variables and interest to advanced users. For these cases, it is best to research the specific model, backend, and runtime, as memory requirements and performance may not follow standard patterns.
The four memory costs that matter
1. Model weights
The model file is the highest obvious cost. A 4-bit quantization uses roughly half a byte per parameter before format and metadata overhead. That means a dense 8B model often lands around 4GB–6GB, while a dense 32B model can require roughly four times as much.
Do not apply the parameter count mindlessly to mixture-of-experts models. Total parameters affect the weight storage and usually the resident weight-memory requirement, while active parameters describe how much computation is used per token. For example, a 12B MoE model with 3B active parameters will still require memory for the full 12B parameter weights, but only 3B are actively used per token during inference. The exact memory behavior depends on how the model and runtime load the experts.
2. Runtime and backend overhead
The runtime needs memory for buffers, kernels, model metadata, tokenization, and other working state. Loading the full model file into a memory pool with no headroom left is therefore not a viable plan.
3. Context and key-value cache
The context window stores the conversation, retrieved passages, system instructions, and generated tokens. Longer context normally increases memory use. A model that works at a modest context can fail or slow sharply as the target context grows.
Start with the runtime’s ordinary default or a deliberately modest target. Increase context only after confirming that the model, speed, and memory pressure remain acceptable.
4. The rest of the computer
The operating system, browser, editor, document index, chat history, and every other process also need memory. Apple unified memory makes this competition especially visible because the CPU and GPU draw from the same pool.
RAM, VRAM, and unified memory
System RAM
System RAM is the main pool for CPU inference and for any model portions that do not fit in dedicated GPU memory. It also holds the operating system and ordinary applications. A CPU-only setup can sometimes load a model that is too large for a GPU, but output may be too slow for comfortable daily use. To speed up CPU inference, reduce context length, choose lighter or more quantized models, and close unnecessary background applications to free up resources.
Dedicated GPU VRAM
Dedicated VRAM is a separate high-bandwidth pool attached to a discrete GPU. It is usually the key constraint for fast local inference on NVIDIA or AMD hardware. A model that fits mostly or entirely in dedicated VRAM generally performs better than one that repeatedly moves data through system memory.
Windows may report “shared GPU memory.” That is system RAM available to the GPU, not additional dedicated VRAM. Do not add it to the dedicated VRAM number and assume the result behaves like a larger graphics card.
Apple unified memory
Apple Silicon combines CPU and GPU access to one memory pool. This avoids a small, fixed VRAM partition, but it doesn't give the model exclusive use of all installed memory. The operating system, applications, model, context, and graphics all compete for the same pool.
A 16GB Apple Silicon Mac is a sensible 7B–8B machine, not a 16GB discrete-GPU equivalent with separate system RAM. A 32GB or 64GB Mac gives substantially more room, but the same headroom rule still applies.
A practical planning formula
For a first-pass text-model estimate, think in this order:
- Start with the actual quantized model file size. Use the specific artifact, not only the family’s parameter count.
- Add runtime overhead. Use the RAM and VRAM calculator rather than assuming the file size is the total.
- Add context overhead. Larger context and multimodal inputs can materially change memory demand.
- Reserve operating-system and application headroom. A desktop machine still needs to function while the model is running.
- Reject a plan that depends on constant swapping. Technical launch is not the same thing as a usable workflow.
Worked examples
8GB Apple Silicon Mac
Start with 1B–4B 4-bit text models. Use short context and close memory-heavy applications. This tier is useful for learning and light tasks, but it is not a comfortable machine for large models or large documents.
16GB Apple Silicon Mac
Start with a 7B–8B 4-bit model. Leave room for macOS and the applications beside the runtime. A 14B model may work in some configurations, but it shouldn't be the default recommendation for a beginner who wants a responsive, general-purpose setup.
Windows PC with 16GB RAM and no discrete GPU
Use a small model and expect CPU-limited speed. More system RAM can help load larger files, but it won't match the throughput of a supported discrete GPU. Choose this path for privacy experiments, short tasks, or learning—not for pretending an office laptop is a high-end inference workstation.
Windows PC with 32GB RAM and 12GB dedicated VRAM
Start with 7B–14B 4-bit models. Plan primarily around 12GB of dedicated VRAM for fast inference, using the 32GB system RAM for the OS, applications, and controlled offload. Watch whether longer context forces meaningful spill into system memory.
64GB Apple Silicon Mac
Start with a 14B–32B 4-bit model based on the job. Higher memory makes larger models practical, but speed and context still decide whether they are useful. Compare output on your real task before keeping the larger model as the default.
Choose a concrete starting model
An existing supported download is more useful than a parameter label alone. These examples use Ollama library tags; in LM Studio, select the corresponding supported GGUF package and inspect its quantization and size.
| Memory situation | First experiment | What changes the choice |
|---|---|---|
| 8GB system or unified memory | Qwen3 1.7B, short context | Operating-system memory pressure can still require a smaller model |
| 16GB CPU-only Windows laptop | Qwen3 4B Q4_K_M, about 2.5GB for Qwen's GGUF file | CPU response speed can limit usefulness before memory fills |
| 16GB Apple Silicon Mac | Qwen3 8B Q4_K_M, about 5.2GB of weights (Ollama's qwen3:8b download; Qwen's own GGUF of this model is a different file at 5.028GB) | Use 4B if other apps or longer context consume the remaining memory |
| Supported NVIDIA GPU with 8GB VRAM | Qwen3 8B Q4_K_M with short context | Display use and runtime overhead can cause spill into system RAM |
| 32GB–64GB system or unified memory | A supported 14B–32B 4-bit text model after a smaller one works | CPU speed, memory bandwidth, context, and quantization remain separate constraints |
Start at 4096-token context for these small text-model experiments, then increase it only when your actual task needs it. See the worked budgets for the 16GB Mac, Windows laptop without a discrete GPU, and 8GB NVIDIA GPU. The memory calculator lets you change the assumptions.
Larger memory: 64GB, 96GB, and 128GB
More capacity enables larger experiments but doesn't guarantee a useful response speed. For perspective, 70 billion parameters at an idealized four bits each require 35 billion bytes, about 32.6GiB, for weights alone. Quantization metadata, tensors stored at other precisions, runtime buffers, context, and normal applications all add to that figure.
On a 64GB unified-memory machine, compare a 14B–32B model with the larger candidate on your real task before making the larger one your default. On a 96GB or 128GB machine, a supported 70B 4-bit model has more room for context and applications, but CPU-only system RAM and fast unified or GPU memory can produce very different speeds. Do not count a multi-GPU setup as one simple memory pool without checking the runtime's split strategy.
For a mixture-of-experts model, “3B active” does not mean its entire download or memory requirement is 3B. Plan from the full selected artifact and the runtime's loading behavior. Similarly, a 120B-class experiment isn't a beginner recommendation just because a compact format fits on paper.
Multimodal and document workflows need extra headroom
Image-capable models may load projectors, vision encoders, or other components in addition to the text model. Document chat adds extraction, embeddings, indexes, retrieved passages, and larger prompts. Image generation can have very different memory behavior from text generation.
Do not use a text-only memory table as a guarantee for these workflows. Start with one small representative input and observe the complete pipeline.
What success looks like
A memory plan is successful when:
- the model loads without exhausting the intended memory pool;
- a normal prompt returns a response at a usable pace;
- context can grow to the intended working size without crashes;
- the operating system and essential applications remain responsive;
- document or multimodal components can run beside the model;
- the setup does not depend on unexplained background swapping.
Common mistakes
Planning from parameter count alone
Parameter count is useful, but the exact artifact, quantization, architecture, context, and runtime determine real memory demand.
Treating 16GB RAM as 16GB VRAM
They are different pools with different bandwidth and different competition from the rest of the system.
Treating unified memory as entirely available to the model
macOS and every other application use the same pool. Reserve headroom.
Maximizing context before the first successful run
Long context can create a memory problem that looks like a model-fit problem. Start modestly, then increase.
Calling a model practical because it launches once
A useful setup must survive normal prompts, growing context, and the applications you need alongside it.
Next step
Use What local LLM can my computer run? For a direct hardware-first path, enter the exact file and context in the RAM and VRAM calculator.