Skip to main content
Local LLM Guide

Model

OLMo 2 1124 13B Instruct

Start with a supported compressed build and short context. 32GB RAM estimate. Best use: Stronger fully open OLMo 2 local chat than the 7B variant

14BText chat

Best use

Stronger fully open OLMo 2 local chat than the 7B variant · Research and education

Memory planning

32GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

Apache-2.0; read the model’s terms before use

Who should use this model?

  • Stronger fully open OLMo 2 local chat than the 7B variant
  • Research and education
  • General chat and summarization on 32GB-class systems
  • Open-science model comparison

Choose something else when

  • You only have 8GB RAM or 8GB VRAM
  • You need supported structured tool calling
  • You need multilingual support beyond primarily English
  • You need vision

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
Memory or response time may limit use. Try a smaller model first.
8GB system memory
Use a smaller model for this memory limit.
16GB system memory
Memory or response time may limit use. Try a smaller model first.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
Use a smaller model for this memory limit.
12GB GPU memory
Memory or response time may limit use. Try a smaller model first.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
14B
Input
Text
Published context limit
4,096 tokens
Model format
Safetensors · GGUF · Ollama
Suggested compression
Q4_K_M for 16GB+ systems; Q5_K_M or Q8_0 only on 32GB RAM, ample unified memory or larger VRAM systems.
Typical Q4 file
8.35GB before runtime and context overhead
Languages
English

First setup

Ollama

ollama run olmo2:13b

LM Studio: Use the official `allenai/OLMo-2-1124-13B-Instruct-GGUF` repository and select Q4_K_M for most local systems.

llama.cpp

Use a standard text GGUF loader against the official Ai2 GGUF repository.

Conditions that change the answer

  • The model is named 13B, but the official GGUF metadata reports model size as 14B.
  • Ollama and release sources describe the OLMo 2 family as trained up to 5T tokens, but the exact instruct checkpoint token count was not separately supported,
  • The official model card describes the language as primarily English.
  • Check that this exact model can use your app’s connected tools before relying on that feature.
  • RAM and VRAM estimates are derived from GGUF/Ollama package size plus conservative overhead; conversation memory, context length, batch size, runtime and GPU offload choices can materially change memory needs.

Read the controlling license text →