Skip to main content
Local LLM Guide

Model

Gemma 3 4B Instruct

Start with a supported compressed build and short context. 12GB RAM estimate. Best use: local multimodal chat

4BMultimodal

Best use

local multimodal chat · image understanding

Memory planning

12GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

Gemma Terms of Use; read the model’s terms before use

Who should use this model?

  • local multimodal chat
  • image understanding
  • structured output
  • function-calling workflows

Choose something else when

  • a text-only runner is required
  • 8GB RAM is the hard ceiling
  • maximum answer quality is required

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB system memory
Memory or response time may limit use. Try a smaller model first.
16GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
12GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
4B
Input
Text · Image · Video
Published context limit
131,072 tokens
Model format
safetensors · GGUF · Ollama · Transformers · Kaggle
Suggested compression
Q4_K_M for default local use; Q5_K_M if the model and multimodal projection fit comfortably in RAM/VRAM.
Typical Q4 file
2.49GB before runtime and context overhead
Languages
English · 140+ languages

Using image input

This model needs an app or library that supports its image-processing components. Follow the exact model’s instructions for preparing image inputs. Some GGUF downloads also need a matching projector file; loading the text model alone does not establish image support.

First setup

Ollama

ollama run gemma3:4b

LM Studio: Search for bartowski/google_gemma-3-4b-it-GGUF and verify image/mmproj support before enabling vision.

llama.cpp

Use hf.co/bartowski/google_gemma-3-4b-it-GGUF:Q4_K_M plus the matching mmproj file for image input.

Conditions that change the answer

  • Video understanding is described by Google for Gemma 3 generally, but local GGUF/Ollama support should be treated as runner-dependent.
  • GGUF files are community conversions, not official Google GGUF releases.
  • The download size includes the Q4_K_M model plus a matching multimodal projection file where applicable.
  • Allow memory for the model file, the app, and your conversation.

Read the controlling license text →