Quick verdict
Start with Qwen3 8B Q4_K_M at 4096-token context on a supported NVIDIA GPU with 8GB dedicated VRAM. Drop to Qwen3 4B if display use, other GPU applications, or the context leaves too little room.
What this profile is for
An 8GB NVIDIA card has a dedicated memory pool separate from system RAM. This example assumes at least 16GB system RAM beside the card. Verify the exact GPU and driver against your runtime's support list; the memory capacity alone does not establish compatibility.
Confirm acceleration before tuning
Run nvidia-smi to confirm the driver sees the card. Follow install Ollama, run ollama run qwen3:8b, and set /set parameter num_ctx 4096 at the chat prompt. While it is loaded, run ollama ps in another terminal. A 100% GPU processor entry means Ollama reports full GPU placement; a split entry means part is in system memory.
For a visual setup, follow install LM Studio, choose a Qwen3 8B GGUF Q4_K_M download, and inspect GPU offload in the load settings. Use the GPU diagnostic if the runtime remains on the CPU.
Worked VRAM example
Assume a 5.2GB model file, a 1.5GB planning allowance for runtime and short context, and 0.8GB already used by the display and other GPU work. That totals 7.5GB of an 8GB pool, leaving only about 0.5GB. These rounded GB assumptions illustrate why this tier can be tight; the allowance is not a measured allocation for Qwen3.
The remaining system RAM is not extra dedicated VRAM. Offloading model layers there can allow a load while slowing generation. Use the calculator with Qwen3 8B selected, enter 8GB dedicated VRAM plus your separate system RAM, and compare the result with the loaded model's actual placement.
When the model spills into system RAM
Close another loaded model or GPU-heavy application, reduce context, and retry the same short question. If it still spills, try qwen3:4b, whose Q4_K_M download is about 2.5GB. Keep the smaller model when its answers solve the task and it leaves enough room for normal use.
A 14B dense 4-bit model is a much tighter fit. Image-capable models and document workflows add components and prompt tokens, so retest the full workload before treating an ordinary text-chat result as sufficient.
Recommended next step
Use RAM and VRAM requirements to understand the two memory pools. Follow out-of-memory troubleshooting for allocation errors and slow-model troubleshooting for a model that loads but feels too slow.