LOCAL_AI_STACK
v2.2 · 2026-06-12
← back to models

Qwen3 4B

qwen3-4b
REASONING

Qwen3 4B is a 4B reasoning model profile for local AI planning. This page records provider, license, context, quantization, hardware-fit estimates, setup hints, source links, and caveats so readers do not choose a model by name alone.

PROVIDER
Qwen
FAMILY
Qwen3
MODEL TYPE
reasoning
PARAMETERS
4B
MODALITIES
text
ARCHITECTURE
Decoder-only Qwen3 causal Transformer with grouped-query attention and a hybrid thinking/non-thinking chat template.
CONTEXT WINDOW
32,768 tokens
TRAINING TOKENS
36.0T
RELEASE DATE
2025-04-29
LICENSE & LINKS
GGUF REPOSITORIES
Official Qwen GGUF quantization repository with Q4_K_M, Q5_K_M, and Q8_0 sizes listed.
SETUP HINTS
OLLAMA
Use `ollama run qwen3:4b` for Ollama's packaged build, or `ollama run hf.co/Qwen/Qwen3-4B-GGUF:Q4_K_M` to pin the official Hugging Face GGUF.
LM STUDIO
Search for `Qwen/Qwen3-4B-GGUF` in LM Studio and select Q4_K_M or Q5_K_M.
LLAMA.CPP
Use `llama-server -hf Qwen/Qwen3-4B-GGUF:Q4_K_M`; use Qwen3 chat template settings and add `/think` or `/no_think` when you need to control thinking mode.
QUANTIZATION FILE SIZES (GGUF)
Q4_K_M
2.5 GB
Q5_K_M
2.89 GB
Q8_0
4.28 GB
Quantization note: Q4_K_M for practical local reasoning; Q5_K_M if memory allows.
RAM / VRAM ESTIMATES
MIN RAM
4.5 GB
COMFORTABLE RAM
6.9 GB
MIN VRAM
4 GB
COMFORTABLE VRAM
5.9 GB
These are conservative local-inference estimates, not official hardware requirements.
HARDWARE FIT
CPU only (no GPU)
yes
8 GB RAM
good
16 GB RAM
good
32 GB RAM
good
8 GB VRAM
good
12 GB VRAM
good
24 GB VRAM
good
Apple Silicon (unified memory)
good_16gb_plus
Hardware fit values are conservative local-inference estimates based on GGUF size plus runtime and KV-cache overhead. Actual requirements depend on context length, quantization, runtime, GPU offload, and other running apps.
BEST FOR
·Small local reasoning model for laptops and constrained machines.
·Fast experimentation with Qwen3 thinking/non-thinking behavior.
·Users who need a compact model with modern Qwen3 architecture.
AVOID IF
·You need consistently strong reasoning on difficult math or code tasks.
·You want a model that never emits thinking traces.
·You need the best quality available on 16GB+ hardware.
CAVEATS
·RAM and VRAM figures are estimates from GGUF file size plus conservative runtime and KV-cache overhead; exact memory depends on context length, batch size, backend, and GPU offload.
·Qwen3 model cards state 32,768 native context and 131,072 tokens with YaRN; this record uses the native context as the primary contextWindowTokens value.
·Reasoning mode may emit thinking content unless disabled by the chat template or `/no_think` prompt convention.
SOURCE URLS
FIELD EVIDENCE
LANGUAGES
ChineseEnglishFrenchSpanishPortugueseGermanItalianRussianJapaneseKoreanVietnameseThaiArabic100+ languages and dialects
CAPABILITIES
tool callingreasoning tuned
← Qwen2.5-Coder 32B InstructQwen3 8B →