Skip to main content
Local LLM Guide

Hardware

Mini PC with 16GB RAM

Use small CPU-friendly models and expect slower output. Avoid large document sets and long context until a small model works well.

Example: 16GB system memoryNo dedicated GPU memory

Model sizes to check

1B-3B · some 4B-7B CPU estimates

Likely bottleneck

CPU throughput and system-memory bandwidth

Good first workload

Small-model chat and light experimentation

Upgrade trigger

Compare a computer with a supported GPU when generation speed remains the problem after testing smaller models.

What this machine is suited for

Use small CPU-friendly models and expect slower output. Avoid large document sets and long context until a small model works well.

  • 1B-3B
  • some 4B-7B CPU estimates

What will limit it

Without a discrete GPU, model loading is only half the problem. CPU inference can be too slow for comfortable daily use even when the model technically fits in RAM.

  • The memory values describe an example configuration; check the actual computer before choosing a model.
  • The estimate changes with the exact model file, context length, runtime, GPU offload, drivers, thermals, and other running apps.
  • A model that technically loads may still be constrained, unstable, or memory-limited for ordinary use. Treat “fits” and “feels good” as different questions.

Before moving to a larger model

  1. Use a common 4-bit build.
  2. Keep context modest until the first run is stable.
  3. Close memory-heavy applications and watch actual memory pressure.
  4. Increase model size only when the smaller model is not solving the task.

What to upgrade

Compare a computer with a supported GPU when generation speed remains the problem after testing smaller models.