Run Local AI on Windows: App Choice and Setup

Leave room for the app, your conversation, and other programs. A model file that fits on disk may still need more memory than your computer has available.

Quick answer: Start with LM Studio for a desktop app, or Ollama for commands, a local API, and connections to other apps. Choose a small downloaded model that fits your dedicated GPU memory and system RAM. Get a short chat working before adding Docker, documents, or Open WebUI. Check support for your exact GPU and driver; a Windows app cannot make an unsupported GPU work or make an oversized model fit.

Beginner recommendation

Start with the row that matches your computer or goal.

The main Windows rule is simple: dedicated VRAM matters more than the big shared-memory number Windows may show. Shared GPU memory is not the same as dedicated graphics memory, and it can be much slower for local AI workloads.

Best Windows setup by hardware class

The point of this table is not to tell you what can be forced to launch. It is to tell you what is a sane first setup for a normal user.

NVIDIA, AMD, integrated graphics, and CPU-only Windows

NVIDIA Windows PCs

NVIDIA is the cleanest beginner path on Windows because CUDA support is mature across the local AI ecosystem. If you have a supported NVIDIA GPU with dedicated VRAM, you can usually choose either LM Studio or Ollama and focus on model size rather than backend complexity.

A practical ladder looks like this:

AMD Windows PCs

AMD on Windows can work, but it is worth checking against the supported-device list for your exact graphics card and app before you start. Support has improved, but it remains more configuration-sensitive than NVIDIA or Apple Silicon.

Check Ollama's current Windows GPU list or the selected LM Studio engine for the exact card and driver. ROCm and Vulkan are different support paths. A card missing from one list may be supported through another path; an advertised VRAM amount alone establishes neither. Once the backend works, choose model weights and context by the same memory calculation used for other GPUs. There is no general rule that every AMD card requires a smaller model class.

Integrated graphics

Integrated graphics can run some local AI workloads, especially with enough system RAM and the right backend. But it should be treated as a low-to-mid expectation path.

Good first uses:

• Small local models.
• Short prompts.
• Learning how local AI works.
• Offline privacy experiments.

Avoid first:

• 32B models.
• 70B models.
• Large PDF collections.
• Long-context workflows.
• Vision or multimodal workflows as a beginner default.

CPU-only Windows

CPU-only local AI is real, but it has sharp limits. It is best for small models, short prompts, offline privacy, and learning. Larger 32B or 70B models can be frustratingly slow on this kind of computer.

If your first run is slow, the app may not be broken. Your machine may simply be running the model on the CPU or spilling into slower shared memory.

High-capacity options: 32GB VRAM (RTX 5090) and 128GB Strix Halo unified memory

Recent hardware releases expand what is possible on a single Windows machine without building multi-GPU server rigs:

• NVIDIA GeForce RTX 5090 (32GB GDDR7): A 32GB card gives a 32B Q4 model more room for conversation history than a 24GB card, but context still consumes memory. A dense 70B model at four-bit precision exceeds 32GB before the conversation cache: for example, Ollama’s DeepSeek-R1 70B Q4_K_M download is about 43GB. Native FP4 support does not make a 70B four-bit model fit in 32GB. Use a smaller model, more graphics memory, a lower-bit build with its own quality tradeoffs, or partial CPU loading; check the exact file and context in the calculator.
• AMD Ryzen AI Max+ 395 Strix Halo (128GB unified memory): The CPU and GPU share one memory pool. The memory available to your model depends on the computer's BIOS graphics allocation, the driver, and the app's runtime; do not count 128GB of RAM and a graphics allocation as two separate pools. Start with a 70B Q4 model and moderate context only after confirming the allocation leaves room for Windows and other programs. On Windows, LM Studio offers a Strix Halo ROCm path; Ollama's Windows ROCm list does not currently include this chip, so check its Vulkan backend and your installed driver instead. In Ollama, run ollama ps after loading the model to check whether it is using the GPU. Capacity alone does not predict generation speed, and fitting a model does not guarantee a fast conversation.

Which Windows path fits your machine?

Use this decision tree.

LM Studio vs Ollama on Windows

Best beginner default: Start with LM Studio if you want a Windows app that feels like an app.

Best for connecting apps: Start with Ollama if you want a runtime that other tools can use.

Best Open WebUI path: Do not start with Open WebUI. First confirm Ollama works locally. Then add Open WebUI.

Your first working Windows conversation

Start with one native Windows app. Check Settings > System > About for Windows version and RAM. In Task Manager's Performance view, identify the GPU and its dedicated memory separately from shared system memory. Ollama's current native Windows path requires Windows 10 22H2 or later; LM Studio's x64 app requires AVX2, and it also offers a Windows ARM build. Use the correct installer for your processor.

With LM Studio

1. Download LM Studio and install the Windows build for your machine.
2. Open Discover and choose a supported instruction model. With 8GB total RAM, start with a small 1B-2B four-bit download. With a 6GB GPU and adequate RAM, a Qwen3-4B GGUF at four-bit compression is a useful first check. MLX downloads are for the app's Apple Silicon engine.
3. Read the full publisher and file name before downloading. Load one model with a modest context, such as 4096 tokens, and open a new local chat.
4. Ask: “Rewrite this plainly without changing the time: The meeting will commence at 10:30.” Confirm that the answer preserves 10:30 and that Windows remains responsive.
5. Quit and reopen the app. Select the same downloaded model and repeat the prompt. Add documents or larger models after this works.

If loading fails, preserve the exact error and follow LM Studio loading checks. A smaller supported download is a better diagnostic than changing several GPU settings at once.

With Ollama

Install Ollama for Windows, start the app, and open a new PowerShell window:

ollama --version
ollama run qwen3:1.7b

The first run downloads the named model if necessary. Ask the same short rewriting question, adding /no_think for this compatible Qwen3 thinking template if desired. Exit with /bye, then run ollama list to confirm the installed model. On a roomier computer, try ollama run qwen3:4b next.

While a model is loaded, run ollama ps in another terminal to inspect CPU/GPU placement and context. A response proves a working conversation, while the processor column tells you whether the intended acceleration is active. If the command itself is missing, use command not found checks; if the model runs on CPU unexpectedly, use GPU checks.

Use the offline guide for a restart test without a connection. A model download needs connectivity, and optional cloud providers or connected tools are separate choices.

Do you need Docker or WSL?

For basic local AI on Windows, usually no.

If you are a beginner, do not make Docker the first problem you solve. First prove that your machine can run a local model.

Driver and OS checklist

Before installing anything, check these items:

• Windows version and edition.
• CPU architecture and instruction support.
• Total system RAM.
• Dedicated GPU model.
• Dedicated VRAM amount.
• Whether Windows is showing shared GPU memory separately.
• NVIDIA or AMD driver status.
• Free disk space on the drive where models will live.
• Whether you want buttons-and-menus, runtime/API, or Open WebUI.
• Whether corporate security software may block installers or local servers.

This checklist matters because Windows local AI problems often look like app problems when they are really hardware, driver, storage, or networking problems.

First model to try on Windows

Start smaller than you think.

Do not use the first model run to prove the largest possible model. Use it to prove the setup works.

Model storage on Windows

Model files can become large quickly. Before downloading many models, decide where you want them stored.

Ollama's standard Windows model folder is %USERPROFILE%\.ollama\models. Use OLLAMA_MODELS to select another directory for the actual Ollama server process. LM Studio's My Models screen lets you choose its models directory. Changing either setting does not automatically prove that existing files were moved; follow the storage guide to copy, restart, and confirm the collection before deleting the old copy.

Practical advice:

• Do not fill your system drive by accident.
• Decide whether models should live on C: or another drive before downloading many large files.
• Keep the first test model small.
• Delete models you do not use.
• Note the folder location so you can find or move your model files later.

What not to try first on Windows

Privacy caveats for Windows local AI

Local AI can be more private than cloud AI, but only for the parts of the workflow that actually remain local.

A Windows setup is more meaningfully local when:

• The model runs on your Windows machine.
• Prompts are processed locally.
• Documents and embeddings stay on local storage.
• The selected provider is not a cloud API.
• The local server is bound only to localhost.
• Cloud features, web search, remote access, and public tunnels are off unless intentionally used.

A Windows local AI setup may still contact the internet for:

• App downloads.
• Model downloads.
• Runtime downloads.
• Update checks.
• Model search.
• Cloud model providers.
• Web search.
• Community hubs.
• Remote access features.
• MCP tools and extensions.

The key rule is this: a local app and a local model are not the same thing. If your local interface is connected to OpenAI, Anthropic, Groq, or another hosted provider, your prompts and uploaded content may leave your computer for inference.

Also remember that local servers can create security issues. A service bound to localhost is very different from a service exposed to your network, a reverse proxy, or a public tunnel. Do not expose Ollama, LM Studio, Open WebUI, or AnythingLLM beyond localhost unless you understand authentication, network access, and the security tradeoff.

Common Windows troubleshooting

Windows beginner setup checklist

Before downloading a model, answer these questions:

• How much system RAM do I have?
• Do I have a discrete GPU?
• How much dedicated VRAM do I have?
• Is my GPU NVIDIA, AMD, or integrated?
• Do I want a desktop app or a runtime/API?
• Do I want Open WebUI eventually?
• Do I want to use PDFs or documents?
• Where should model files be stored?
• Am I handling sensitive data?
• Am I willing to use Docker or WSL?

Then choose:

Frequently asked questions

Can Windows run local AI?

Yes. Windows can run local AI with tools such as LM Studio, Ollama, and Open WebUI. The quality of the experience depends heavily on RAM, GPU, dedicated VRAM, drivers, model size, and whether the workload stays on GPU or falls back to slower memory.

Is Ollama available on Windows?

Yes. Ollama has a Windows install path. It is a good choice if you want a local runtime, local API, or a backend for tools like Open WebUI.

Should I use LM Studio or Ollama on Windows?

Use LM Studio if you want the easiest desktop app. Use Ollama if you want a runtime/API or plan to add Open WebUI or other integrations.

Do I need Docker for local AI on Windows?

Not for basic LM Studio or basic Ollama. Docker usually enters the picture when you want tools like Open WebUI or self-hosted interface layers.

Do I need WSL?

Not for basic LM Studio or basic Ollama. WSL is often part of Windows workflows for Docker-based or Linux-oriented tools, including many Open WebUI tutorials.

Can I run local AI without a GPU?

Yes, but expectations should be modest. CPU-only or integrated-graphics setups should start with small models and short prompts.

Where does Ollama store models on Windows?

The standard desktop installation stores them in %USERPROFILE%\.ollama\models. A custom server or another account can use a different folder. Follow the storage guide if you need to move the collection.

Why is local AI slow on my Windows PC?

Common reasons include using a model that is too large, running on CPU, spilling into shared memory, outdated drivers, too much context, or not enough dedicated VRAM.

What to read next

• What Is Local AI?
• Ollama vs LM Studio
• How to Install Ollama
• How to Install LM Studio
• How to Install Open WebUI with Ollama
• Best Local AI for 8GB RAM
• Best Local AI for 16GB RAM
• Best Local AI for 32GB RAM
• Is Local AI Actually Private?

Sources

• Ollama Windows requirements and storage
• Ollama GPU support and processor placement
• LM Studio system requirements and Strix Halo runtime release note
• Qwen3-1.7B, Qwen3-4B, and DeepSeek-R1 70B download size