What can this machine run?
Start with the memory pool that will hold the model, then leave room for context, the operating system, and normal work. These profiles give a direct starting model class and the bottleneck to expect.
Machine recommendations
Start with your computer’s memory.
Apple unified memory, dedicated GPU memory, and ordinary system RAM are different constraints. Pick the closest machine below, then use the calculator for an exact model file and context target.
Apple Silicon
Mac with 8GB unified memory
Use 1B–4B 4-bit models, keep context short, and close memory-heavy applications.
- System memory
- 8GB
- Memory type
- Unified memory
- Model range
- 1B-3B and very small Q4
- Likely bottleneck
- One shared pool for the model, context, macOS, and applications
Mac with 16GB unified memory
Start with 7B–8B 4-bit models. Move larger only when the smaller model is not solving the task.
- System memory
- 16GB
- Memory type
- Unified memory
- Model range
- 1B-3B · 4B-7B · some 8B-14B Q4
- Likely bottleneck
- One shared pool for the model, context, macOS, and applications
Mac with 32GB unified memory
Start with 8B–14B 4-bit models and preserve headroom for context and normal applications.
- System memory
- 32GB
- Memory type
- Unified memory
- Model range
- 4B-7B · 8B-14B · some 15B-34B Q4
- Likely bottleneck
- One shared pool for the model, context, macOS, and applications
Windows without a discrete GPU
Windows laptop with no discrete GPU
Use small CPU-friendly models and expect slower output. A supported graphics card can help when the processor is too slow.
- System memory
- 16GB
- GPU memory
- No discrete GPU memory
- Model range
- 1B-3B · some 4B-7B CPU
- Likely bottleneck
- CPU throughput and system-memory bandwidth
Mini PC with 32GB RAM
Use small CPU-friendly models and expect slower output. A supported graphics card can help when the processor is too slow.
- System memory
- 32GB
- GPU memory
- No discrete GPU memory
- Model range
- 1B-3B · 4B-7B · some 8B-14B CPU
- Likely bottleneck
- CPU throughput and system-memory bandwidth
AMD Strix Halo 128GB unified memory
Start with 8B–14B 4-bit models and preserve headroom for context and normal applications.
- System memory
- 128GB
- Memory type
- Unified memory
- Model range
- 32B · 70B · some 120B Q4
- Likely bottleneck
- One shared pool for the model, context, macOS, and applications
NVIDIA GPU tiers
NVIDIA 8GB VRAM GPU
Start with 7B–8B 4-bit models and moderate context. Larger models will usually spill into slower system memory.
- System memory
- 32GB
- GPU memory
- 8GB
- Model range
- 7B/8B Q4
- Likely bottleneck
- Dedicated GPU memory, then context length
NVIDIA 12GB VRAM GPU
Use 7B–14B 4-bit models. Keep enough GPU memory for context and runtime overhead.
- System memory
- 32GB
- GPU memory
- 12GB
- Model range
- 7B/8B and some 14B Q4
- Likely bottleneck
- Dedicated GPU memory, then context length
NVIDIA 16GB VRAM GPU
Start around 14B 4-bit. Test context growth before moving to larger weights.
- System memory
- 32GB
- GPU memory
- 16GB
- Model range
- 14B class
- Likely bottleneck
- Dedicated GPU memory, then context length
NVIDIA 24GB VRAM GPU
Start with 14B–32B 4-bit models. Larger experiments still depend on context, offload, and system memory.
- System memory
- 64GB
- GPU memory
- 24GB
- Model range
- 14B to 32B · 70B requires CPU offload or a smaller-than-Q4 file
- Likely bottleneck
- Dedicated GPU memory, then context length
Desktop with 32GB VRAM GPU
Start with 14B–32B 4-bit models. Larger experiments still depend on context, offload, and system memory.
- System memory
- 64GB
- GPU memory
- 32GB
- Model range
- 14B to 32B Q4 with context headroom
- Likely bottleneck
- Dedicated GPU memory, then context length
RAM, GPU memory, and unified memory are not interchangeable.
System RAM
Your computer’s main memory holds the operating system, apps, and any model work that is not in dedicated graphics memory.
Dedicated GPU memory
A separate graphics card has its own memory. The model usually runs faster when its work fits there.
Unified memory
Apple Silicon and AMD Strix Halo share a memory pool across the CPU, GPU, operating system, and apps. GPU allocation and runtime support still matter.