Skip to main content
Local LLM Guide

Hardware

8GB laptop

The GPU and its memory are not supplied for this example. Check those details before choosing a GPU model; a small CPU model can be a first test.

Example: 8GB system memoryGPU memory not supplied

Model sizes to check

Small Q4 models only

Likely bottleneck

Check the GPU and its memory; system RAM alone does not establish GPU fit

Good first workload

Short chat, rewriting, and small summaries

Upgrade trigger

Check the GPU and its memory before deciding which hardware to change.

What this machine is suited for

The GPU and its memory are not supplied for this example. Check those details before choosing a GPU model; a small CPU model can be a first test.

  • Small Q4 models only

What will limit it

This profile does not specify a GPU or a GPU-memory amount. Check the graphics processor, its memory, and support in your chosen app before treating a RAM estimate as a GPU fit.

  • The graphics processor and its memory are not supplied for this example. System RAM alone does not establish a GPU fit.
  • The memory values describe an example configuration; check the actual computer before choosing a model.
  • The estimate changes with the exact model file, context length, runtime, GPU offload, drivers, thermals, and other running apps.
  • A model that technically loads may still be constrained, unstable, or memory-limited for ordinary use. Treat “fits” and “feels good” as different questions.

Before moving to a larger model

  1. Use a common 4-bit build.
  2. Keep context modest until the first run is stable.
  3. Close memory-heavy applications and watch actual memory pressure.
  4. Increase model size only when the smaller model is not solving the task.

What to upgrade

Check the GPU and its memory before deciding which hardware to change.