Quick diagnosis
A model running slowly does not establish that the GPU is missing. Check whether the operating system sees the graphics card, whether the runtime supports it, and whether this particular loaded model uses it.
First three checks
- Identify the exact GPU and driver; on an NVIDIA machine run
nvidia-smi. - Start a short Ollama chat, then run
ollama psin a second terminal and inspect the PROCESSOR column. - Compare your GPU and operating system with the runtime's supported hardware before changing offload settings.
Check the operating system first
For NVIDIA, nvidia-smi should show the GPU name, driver, and memory information. If the command is missing or cannot communicate with the driver, fix the NVIDIA driver installation first and restart as required. This command does not apply to Apple or AMD graphics.
For an Apple Silicon Mac, use About This Mac to confirm the chip. The CPU and GPU share unified memory; there is no separate NVIDIA VRAM pool. For AMD, match the exact card and operating system against Ollama's hardware support list. A familiar brand name is not enough to establish backend support.
Check a loaded model
Start a small downloaded model with ollama run MODEL_NAME, ask a short question, and run ollama ps while it remains loaded. Replace the placeholder with the exact name from ollama list.
| Result | Meaning | Next action |
|---|---|---|
| 100% GPU | Ollama reports the model loaded on the GPU | Investigate context, model size, and heat if generation is still slow |
| A CPU/GPU split | Only part of the model is on the GPU | Close other GPU workloads, shorten context, or try a smaller model |
| 100% CPU | This loaded model is using the CPU | Check support, driver visibility, and runtime logs |
| No listed model | Nothing is currently loaded, or it has already unloaded | Ask another question and check again immediately |
In LM Studio, inspect the model's load settings and GPU offload control, then load a smaller supported model. A GPU visible in Windows Task Manager does not by itself prove that the runtime selected it.
If Ollama runs in Docker
Check which container runs the model. Giving GPU access to the Open WebUI container does not give it to a separate Ollama container. Confirm the NVIDIA container runtime is installed on the host and that the Ollama container was started with the intended GPU access. First get the same small model working on the host; then troubleshoot container access as a separate step.
If acceleration disappears after sleep, restart Ollama and retry the small-model check. Persistent discovery errors belong in the runtime log and driver diagnosis, rather than in prompt or temperature settings.
When to stop
If your GPU has no supported backend on this operating system, use a small CPU model or a supported runtime. Do not replace drivers on a managed machine without its administrator's involvement.
Use install Ollama or install LM Studio for the setup path. Once GPU use is confirmed, follow slow model troubleshooting or the memory calculator.