Start with native Ollama and one small model. This is a practical first path for a Linux user who can open a terminal and wants local chat or a connection other apps can use. Docker, a browser workspace, and a coding agent can wait until that first conversation works.
For a graphical app, LM Studio is another option. Its Linux download is an AppImage; check its current distribution and CPU requirements rather than assuming every Linux desktop is supported. The steps below use Ollama on a system with systemd, the service manager used by many common distributions. A distribution without systemd needs the manual startup path in Ollama's Linux documentation.
Check the computer first
Note your system RAM, available disk space, CPU architecture, and exact graphics card. Download the build intended for your architecture; an x86-64 program is not the native ARM64 build.
An NVIDIA or supported AMD GPU can help, but you can begin with a small CPU model. Use the RAM and VRAM calculator before a large download. A model's download size leaves out conversation memory, temporary calculations, and memory for the operating system.
Install the model runner
Ollama documents this installation command:
curl -fsSL https://ollama.com/install.sh | sh
It downloads and runs an installation script. If you want to inspect the script before running it, open the linked address first; the official Linux page also provides manual installation steps. The installer may request permission to change system files and configure a service.
Open a new terminal and run:
ollama --version
systemctl status ollama --no-pager
The first command should print a version. The second should show the service running. If it is installed but stopped, use sudo systemctl start ollama, then check its status again. For a deliberate manual installation, start ollama serve in a separate terminal instead. Do not start a second server when a service is already using the port.
Download one model and have a conversation
Use a small starting model such as Qwen3 1.7B:
ollama pull qwen3:1.7b
ollama run qwen3:1.7b
The first command needs internet access and free disk space. After the download, ask: “Rewrite this sentence in plain English: Please advise as to your availability tomorrow. /no_think” The /no_think instruction is specific to Qwen3's supported thinking mode; it is not a universal model command.
A useful first result is a completed, sensible answer while your desktop remains responsive. Exit the chat with /bye. Run ollama list to confirm the download remains available, then reopen the same model. This separates a successful temporary chat from a model you can use again.
Check whether the GPU is actually used
During a loaded conversation, open a second terminal and run ollama ps. Its processor column distinguishes CPU, GPU, and a split between them. A large amount of RAM or a working nvidia-smi command does not by itself show that the model uses the GPU.
For NVIDIA, nvidia-smi can establish that the driver sees the card. For AMD, compare the exact card and driver with Ollama's hardware documentation. Backend and operating-system support matter independently of how much memory the card has. Follow GPU troubleshooting if the model falls back to CPU.
If the first run fails
| Symptom | First useful action |
|---|---|
ollama is not found | Reopen the terminal and use the command-not-found guide |
| The command cannot reach the service | Check systemctl status ollama; read journalctl -u ollama --no-pager -n 100 for the failure |
| The download fails | Check disk space and connection using the download guide |
| The model loads but runs slowly | Try a smaller model and a shorter conversation; inspect ollama ps |
| Loading reports insufficient memory | Reduce the selected context and follow memory troubleshooting |
Do not delete the model library or reinstall graphics drivers merely because one model is too large. Preserve the error and diagnose the failing step.
Continue from the working chat
Try offline use before relying on the setup away from the internet. Add Open WebUI if you want a browser workspace, or follow CPU-only guidance if this machine has no usable GPU. Each additional app brings its own storage, network, and provider settings.