You can run a local language model on a CPU. Start with a small quantized model and decide whether its answers and waiting time are useful for your task. Extra RAM may let a larger model load, but it does not make an ordinary CPU generate answers as quickly as a suitable GPU.
This suits learning, short offline writing, simple extraction from supplied text, and occasional chat on an existing computer. Long documents, large models, and coding agents can make the wait impractical. A supported integrated GPU may be another option, but this guide does not count shared memory as dedicated VRAM.
Choose the app that matches the computer
- Ollama: useful if you can use a terminal and want local model commands or an API. Check its platform requirements.
- LM Studio: useful if you want menus and a desktop chat. Its x64 Windows app requires AVX2 CPU instructions; an old processor is not supported merely because it runs Windows.
- llama.cpp: useful if you need direct CPU control and can manage the build or appropriate binaries. Its build guide documents CPU builds; this is a more technical path.
Choose one for the first test. Installing several apps can duplicate model downloads without making the CPU faster.
Start with a model that leaves room for everything else
For an 8GB RAM computer, investigate a 1B–2B-class 4-bit model first. For 16GB or more, a 3B–4B candidate may be worth comparing once the small model works. These are starting choices, not measured performance tiers; the CPU, memory bandwidth, context, and background applications matter.
In Ollama, an example small download is:
ollama run qwen3:1.7b
That tag names a particular packaged model. The first run downloads it; later local runs can reuse it. Check its exact file and license before choosing it for work.
Enter system RAM, the model, and the intended context in the memory calculator. Leave room for your operating system, editor, browser, and temporary model calculations. A model file fitting in RAM does not establish the whole workload fits.
Test one realistic short task
Close unnecessary memory-heavy apps. Start a fresh conversation and supply a short paragraph you understand. For example:
List the two actions requested in this note without adding any: “Send the draft on Tuesday. Call the client after the draft is sent.” /no_think
For the original Qwen3 model, /no_think can request a response without extended thinking when the app's model configuration permits it. Other models use different controls.
Judge both the answer and the wait. A useful result names the two actions, preserves their order, finishes in a time you can tolerate, and leaves the desktop usable. Repeat with your own harmless material; a greeting is a weak test of a real writing or coding workload.
In Ollama, run ollama ps while it is loaded. CPU-only processing is expected on this path; do not treat it as an error requiring driver installation.
Make the workload smaller without changing its purpose
Keep prompts focused and use the context you actually need. Work with a section at a time when that still answers the question. If a task requires comparing provisions across an entire document, splitting it blindly may hide the necessary relationship; use a document-search workflow or accept that this machine may be unsuitable.
Avoid loading several models at once. Give a longer prompt time to finish processing before deciding that output has frozen. For a laptop, compare performance while plugged in and pay attention to heat and fan behaviour; power limits can affect sustained work.
When to stop tuning
Heavy disk swapping, repeated memory errors, or a desktop that stalls indicate that the workload is too large. Try the smaller model or shorter context first. If it loads but the answers are inadequate, check model and prompt quality. More threads or a larger context is not a general cure.
For direct llama.cpp use, consult your binary's --help for thread and GPU-layer controls. CPU settings should be tested with your own task; maximum thread count is not automatically the best setting.
If a small model remains too slow or inaccurate for your required job, the honest result is that this setup is not useful for that job. Use a more capable computer or a deliberately chosen remote service after considering privacy and cost. The cloud vs local cost calculator can explore costs, but it does not establish equivalent quality.