Skip to main content
Local LLM Guide

Hardware

Mac with 32GB unified memory

Start with 8B–14B 4-bit models. Larger models become practical only when enough unified memory remains after the operating system and context.

Example: 32GB unified memory

Model sizes to check

4B-7B · 8B-14B · some 15B-34B Q4 estimates

Likely bottleneck

Shared memory used by the model, context, operating system, and other applications

Good first workload

Chat, coding, and moderate document workflows

Upgrade trigger

Move to more unified memory when normal applications or longer context force repeated swapping or model downsizing.

What this machine is suited for

Start with 8B–14B 4-bit models. Larger models become practical only when enough unified memory remains after the operating system and context.

  • 4B-7B
  • 8B-14B
  • some 15B-34B Q4 estimates

What will limit it

Unified memory is flexible, but it is not free GPU memory. Every open application and every increase in context uses the same pool as the model.

  • The memory values describe an example configuration; check the actual computer before choosing a model.
  • The estimate changes with the exact model file, context length, runtime, GPU offload, drivers, thermals, and other running apps.
  • A model that technically loads may still be constrained, unstable, or memory-limited for ordinary use. Treat “fits” and “feels good” as different questions.

Before moving to a larger model

  1. Use a common 4-bit build.
  2. Keep context modest until the first run is stable.
  3. Close memory-heavy applications and watch actual memory pressure.
  4. Increase model size only when the smaller model is not solving the task.

What to upgrade

Move to more unified memory when normal applications or longer context force repeated swapping or model downsizing.