With 6GB of dedicated graphics memory, begin with a supported 3B–4B model at a 4-bit setting and a short conversation. A 7B–8B 4-bit model is an option to investigate after that first run, but its weights, conversation memory, and temporary calculations may leave little room or force some work onto the CPU.
This suits a desktop or laptop with a supported discrete GPU and enough system RAM for the app and operating system. It is a useful starting point for chat, short writing tasks, and small coding questions. It is not a comfortable default for large document collections or a coding agent carrying many files in each request.
Confirm what the 6GB number describes
On Windows, open Task Manager, choose Performance, and inspect the relevant GPU's dedicated memory. Shared GPU memory is borrowed from system RAM; do not add it to dedicated VRAM and treat the total as one fast pool.
Record the exact GPU and operating system. A 6GB card needs a model runner that supports its driver and backend. NVIDIA, AMD, and Intel support differ by app and platform; a card's capacity alone does not establish that Ollama or LM Studio can accelerate it. Use the GPU troubleshooting guide if that support is unclear.
Start with an exact, smaller download
For Ollama, one practical candidate is Qwen3 4B:
ollama run qwen3:4b
Ollama's model page identifies this packaged download as Q4_K_M and about 2.5GB on disk. That leaves more space than an 8B download, but the file size is not its total VRAM requirement. In LM Studio, select a supported Qwen3 4B GGUF at Q4_K_M and check the publisher and file details; do not substitute an MLX download on a Windows GPU.
Use the RAM and VRAM calculator with the selected model, compression, dedicated VRAM, system RAM, and an initial context such as 4096 tokens. The answer is a planning estimate. Other GPU users, including the desktop display, can reduce available memory further.
Check the first conversation
Load only one model. Ask it to rewrite a short sentence, then ask a small follow-up. In Ollama's chat, /set parameter num_ctx 4096 sets a modest context for a subsequent request. Open another terminal and run ollama ps while the model is loaded.
You want a completed answer and a responsive computer. The processor column reveals whether the model is using GPU, CPU, or a split. A split can be usable, but it is different from a model fitting entirely into dedicated VRAM. Do not interpret a completed answer as proof of full GPU use.
In LM Studio, inspect the load settings, GPU offload, and displayed memory estimate. If loading fails, shorten context before repeatedly retrying the same large allocation. Follow LM Studio loading checks for an unsupported file or model-runner problem.
When to try 7B or 8B
Try a larger file when the smaller model works, the calculator leaves acceptable memory, and its extra capability helps your actual task. Keep the conversation short at first. Q4, Q5, and Q8 versions of the same model have different weight sizes; a 7B or 8B label alone does not determine fit.
If the larger model requires CPU loading, compare one real prompt with the smaller model. Decide whether the answer improvement is worth the wait. There is no hardware-independent tokens-per-second figure that can settle that choice.
Symptoms that change the recommendation
| Symptom | Next action |
|---|---|
| Out of memory at load | Shorten context, unload another model, or select the smaller file |
| Fine in a new chat, then slow later | Recheck accumulated context and conversation memory |
| CPU-only processing | Confirm the exact GPU, driver, and selected model-runner support |
| Large PDF chat fails | Separate document extraction and search from model capacity |
| Wrong answers despite a successful load | Follow answer-quality troubleshooting; memory fit is not accuracy |
If you already own the card, try this workflow before buying a replacement. If you are shopping specifically for larger models, use the GPU selection guide to plan around the intended model and context instead of the card's gaming reputation.