Windows laptop with 8GB VRAM GPU

What this machine is suited for

Start with 7B–8B 4-bit models and moderate context. Larger weights may need system memory if the selected app supports CPU offload.

• 4B-7B
• some 8B-14B Q4 estimates

What will limit it

Fast local inference usually depends on keeping most model work in GPU memory. If the app moves work into system RAM, responses can become slower.

• The memory values describe an example configuration; check the actual computer before choosing a model.
• The estimate changes with the exact model file, context length, runtime, GPU offload, drivers, thermals, and other running apps.
• A model that technically loads may still be constrained, unstable, or memory-limited for ordinary use. Treat “fits” and “feels good” as different questions.

Before moving to a larger model

1. Use a common 4-bit build.
2. Keep context modest until the first run is stable.
3. Close memory-heavy applications and watch actual memory pressure.
4. Increase model size only when the smaller model is not solving the task.

What to upgrade

Move to more GPU memory when the model or context repeatedly spills into system RAM.