How to Run Llama Locally

Quick answer

Running Llama locally means the model weights live on your own machine and text generation happens on your own CPU, GPU, or unified memory instead of a hosted API. In practice, the easiest routes are Ollama for a fast CLI or local server and LM Studio for a desktop GUI. For most people, the practical Llama starting points today are Llama 3.2 1B or 3B, Llama 3.1 8B, and, on high-memory systems, Llama 3.3 70B. Llama 4 Scout and Llama 4 Maverick are real local options only on workstation or server-class hardware, not ordinary laptops. As a simple rule, start with 3.2 3B on 8GB to 16GB, 3.1 8B on 16GB or more, and 3.3 70B only when you have roughly 64GB-class memory or substantial VRAM.

Pick the right Llama family

"Run Llama locally" does not require using Meta's raw checkpoints directly. It means you are running a Llama-family model on your own computer, usually through a local runtime such as Ollama, LM Studio, or llama.cpp. Once LM Studio has the model on your device, chatting and document processing can stay offline and on-device, and Ollama serves its API on localhost by default.

Meta's current official Hugging Face organization lists Llama 4 as the current family and Llama 3.3, 3.2, 3.2 Vision, and 3.1 in history. For new local installs, those are the generations that matter most; older Llama 3 and Llama 2 releases are legacy choices now unless you need them for compatibility or a specific fine-tune.

For most local users, the sweet spot is still dense text models first, multimodal and MoE later. That usually means Llama 3.2 3B for lightweight laptops, Llama 3.1 8B for the broadest "good enough" general-purpose experience, Llama 3.2 Vision 11B when you need image input, and Llama 3.3 70B when you have serious memory and want a noticeable jump in text quality.

Model selection table

A practical takeaway: if your goal is simply "I want Llama on my computer today," the best first downloads are usually Llama 3.2 3B or Llama 3.1 8B. If your goal is "I want the strongest Llama I can realistically run," then Llama 3.3 70B is the high-end dense target, while Llama 4 is best treated as a workstation-class project.

When a non-Llama model is the better pick

Choose a non-Llama family when your constraint is not "I want Meta's model," but rather memory, license simplicity, or specialized capability. Google's Gemma 4 docs publish explicit inference memory estimates as low as about 3.2GB in Q4_0 for Gemma 4 E2B and 5GB for Gemma 4 E4B, making them especially attractive on modest hardware. Microsoft's Phi family is explicitly pitched for on-device use and is available under the MIT License. For software engineering and agentic coding, Mistral's official Devstral Small 2 24B card says it is light enough to run on a single RTX 4090 or a Mac with 32GB RAM and uses Apache 2.0.

The "best local model" is often not a Llama. If you need the smallest competent local model, try Gemma or Phi. If you mainly code, look hard at Devstral. If you specifically want the Meta ecosystem, broad community familiarity, and many local packaging options, Llama still makes sense.

Fastest path with Ollama

Ollama is the fastest way to go from zero to "a local Llama is answering prompts." It supports Apple GPUs through Metal, NVIDIA GPUs on supported drivers and compute capability, AMD support through ROCm on supported configurations, and experimental Vulkan support on Windows and Linux.

Install Ollama

On macOS and Linux, the official install script is the fastest path. On Windows, Ollama provides a PowerShell installer and also ships a native Windows app. macOS requires macOS 14 Sonoma or newer; the macOS docs say Apple M-series systems get CPU and GPU support, while x86 Macs are CPU-only. Windows support starts at Windows 10 22H2 or newer.

# macOS or Linux
curl -fsSL https://ollama.com/install.sh | sh

# Windows PowerShell
irm https://ollama.com/install.ps1 | iex

Pull a first Llama model

For most people, one of these is the right first command:

# safest beginner choice
ollama run llama3.2:3b

# better quality if you have more memory
ollama run llama3.1:8b

# only for high-memory systems
ollama run llama3.3

If you are unsure, use this sequence: 8GB machine → llama3.2:3b, 16GB+ machine → llama3.1:8b, 64GB-class machine → consider llama3.3.

Check whether Ollama is using your GPU

After the model is loaded, run:

ollama ps

The PROCESSOR column shows whether the model is 100% GPU or is being split or pushed onto CPU. Avoiding CPU offload usually gives the best performance.

Lower context when memory is tight

Ollama's docs say larger context lengths increase memory usage. They also publish default context behavior based on available VRAM: under 24 GiB VRAM → 4K context, 24–48 GiB → 32K, and 48 GiB or more → 256K. On consumer hardware, lowering context is one of the fastest fixes for load failures and sluggishness.

OLLAMA_CONTEXT_LENGTH=8192 ollama serve

Import your own GGUF

If you eventually want to use a Hugging Face GGUF that is not already in Ollama's public library, create a Modelfile with a line such as FROM /path/to/file.gguf, then build it with ollama create. Ollama can also quantize FP16 or FP32 models into formats such as q8_0, q4_K_S, and q4_K_M.

GUI path with LM Studio

LM Studio is the easiest GUI if you want search, download, load, and chat in one desktop app. The docs say LM Studio runs local LLMs on macOS, Windows, and Linux using llama.cpp, and on Apple Silicon Macs it also supports MLX. The app can also operate offline once you already have model files on disk.

Install LM Studio and confirm your platform fit

LM Studio's system requirements say Apple Silicon Macs only on macOS, with 16GB+ RAM recommended and 8GB Macs limited to smaller models and modest context sizes. On Windows, LM Studio supports x64 and ARM, requires AVX2 on x64, recommends at least 16GB RAM, and recommends at least 4GB dedicated VRAM. Linux support includes x64 and ARM64, with Ubuntu 20.04 or newer.

Download the model in the app

LM Studio's built-in downloader can search Hugging Face by keyword, exact repository name, or even a full Hugging Face URL. In the app, go to Discover, search for llama, and download a supported model that matches your hardware. The same advice applies: 3B first, then 8B, then 70B only on large-memory systems.

Load and chat

LM Studio's getting-started docs explain that "loading" a model means allocating memory so your computer can hold the model's weights and parameters in RAM. On Apple Silicon, prefer an MLX build when an MLX package is available; on other platforms, GGUF through llama.cpp is the normal default.

Import a GGUF you downloaded elsewhere

If you already obtained a GGUF file outside the app, LM Studio supports importing it:

lms import /path/to/model.gguf

The docs say imported models are stored in LM Studio's models directory using a publisher/model/file structure.

Start the local server when you need an API

LM Studio can act as a local API server from the Developer tab or by CLI:

lms server start

The lms CLI ships with LM Studio. The REST load endpoint supports context length, Flash Attention, MoE expert count, and KV-cache offload settings.

Understand formats, hardware, and speed

Official weights, Hugging Face access, GGUF, quantization, and Ollama packages

For most readers, the practical rule is simple: if you want the least friction, use Ollama packages or LM Studio downloads; if you want ecosystem flexibility, use Hugging Face Transformers; if you want the classic local LLM path, use GGUF in llama.cpp-based tools.

GGUF and quantization in plain English

GGUF is the local-LLM format you see most often in desktop tools because it is meant for inference and for GGML-based runtimes. Quantization means storing model weights in fewer bits so the file is smaller and the runtime can use less memory. The tradeoff is that lower-bit versions are not identical in quality to BF16 or FP16 originals. Ollama's import docs state this plainly: quantization lets you run models faster and with less memory consumption but at reduced accuracy.

That is why one "Llama 8B" can feel very different from another "Llama 8B." A BF16 checkpoint, a Q8 GGUF, and a Q4_K_M Ollama package are all effectively different deployment forms of the same base model. They may differ in load behavior, speed, memory fit, and sometimes answer quality.

Cautious hardware ranges

On Apple Silicon, the local story is unusually strong because Ollama uses Metal and LM Studio supports MLX in addition to llama.cpp. The practical caution is still memory: LM Studio recommends 16GB+, and says 8GB Macs should stay with smaller models and modest contexts.

On Windows with NVIDIA, Ollama says supported NVIDIA cards need compute capability 5.0+ and driver version 531 or newer. LM Studio recommends 4GB or more of dedicated VRAM and an AVX2 CPU on x64 Windows.

Licensing and access

At a practical level, the first thing to know is that Llama is not distributed under Apache 2.0 or MIT. Meta uses a custom community license plus an acceptable use policy for each release family. Requests for Hugging Face access are processed hourly after you accept the license and acceptable use policy.

For many users, the second practical point is redistribution. The Llama 3.1 and 3.2 license texts say that if you distribute Llama materials or a product or service containing them, you must include a copy of the agreement, retain the attribution notice, and prominently display "Built with Llama." The same license text also says that if you use Llama materials or outputs to create or improve a distributed AI model, you must include "Llama" at the beginning of that model's name. The current 3.1 and 3.2 licenses also contain an additional commercial term for organizations above 700 million monthly active users.

There are also release-specific access caveats. The Hugging Face page for Llama 3.2 Vision states that, for the multimodal models in that release, the rights granted under the 3.2 community license are not granted to individuals domiciled in the EU or companies with their principal place of business in the EU, except for end users of products or services that incorporate those multimodal models.

At a high level, the acceptable use policy forbids categories such as unlawful activity, harmful or abusive conduct, unauthorized professional practice, malware, fraud, impersonation, and certain dangerous use cases. Read the exact license and AUP for the specific model and version you plan to ship, because these terms are versioned and can change.

Troubleshooting and FAQ

When the model is slow, refuses to load, or runs out of memory

The first fix is almost always to use a smaller model, a smaller quant, or a shorter context. Ollama states that context length directly increases memory use, and LM Studio exposes context length, Flash Attention, and other load parameters that can materially change fit and speed.

The second fix is to check whether the model is actually on the GPU. In Ollama, ollama ps shows whether the model is fully GPU-resident or partly using CPU. If a big model is split between GPU VRAM and system RAM, it may load, but it will usually feel much slower than a smaller model that fits cleanly.

The third fix is to inspect logs instead of guessing. Ollama documents where logs live on macOS, Linux, and Windows, and supports debug logging on Windows through OLLAMA_DEBUG=1.

If you are on Linux with NVIDIA and Ollama suddenly falls back to CPU after sleep or resume, Ollama's GPU docs call out a known suspend/resume failure mode and suggest reloading the NVIDIA UVM driver.

If the model loads but answers badly, especially a sideloaded GGUF, suspect prompt template mismatch before assuming the model is "bad." LM Studio says it auto-configures prompts from metadata and lets you override them. A wrong or missing chat template can make a good model look broken.

If you are downloading official Meta weights and get a 403, do not assume access was revoked. Meta's own README says the signed download links expire after 24 hours and after a certain number of downloads, and that you can request a fresh link.

FAQ

Can I run Llama without a GPU?

Yes. llama.cpp supports CPU builds, and Ollama's macOS docs explicitly note x86 Macs are CPU-only. But the practical experience is much better with a GPU or Apple unified-memory machine, especially once you move beyond 3B and 8B models.

Is Ollama or LM Studio easier for a beginner?

If you want the fewest steps and do not mind a terminal, Ollama is the fastest path. If you want a GUI, search, load settings, and easy local serving in one app, LM Studio is usually friendlier.

Do I need official Meta weights to say I am running Llama locally?

No. In practice, many users run Llama locally through GGUF builds in llama.cpp-based apps or through packaged Ollama models.

What is GGUF in plain English?

GGUF is a local-inference file format used by GGML-based runtimes such as llama.cpp. It is designed for fast loading, single-file deployment, and easy local distribution.

Why does a 128K-context model still fail on my computer?

Because the advertised maximum context is not the same as a practical local context. Ollama says memory requirements rise with context length and even lowers default context allocations based on available VRAM. A model may "support 128K" on paper while your machine only runs it comfortably at 4K, 8K, or 32K.

Which Llama should I choose on a 16GB machine?

Usually Llama 3.1 8B if you want the best general-purpose Llama experience that still feels reasonable, or Llama 3.2 3B if you care more about speed and lower memory pressure. On 16GB, 70B is not the right starting point.

Can I use Llama commercially?

At a high level, yes, Meta's licenses are written for commercial and research use, but they are custom Llama Community Licenses, not standard permissive licenses. You still need to follow the license, attribution, AUP, and any release-specific or scale-specific terms that apply to your version.