Skip to main content
Local LLM Guide

Model

IBM Granite 4.1 8B

Start with a supported compressed build and short context. 24GB RAM estimate. Best use: Enterprise local chat

8.79BText chat

Best use

Enterprise local chat · document search

Memory planning

24GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

Apache-2.0; read the model’s terms before use

Who should use this model?

  • Enterprise local chat
  • document search
  • Structured JSON generation
  • Tool calling
  • Coding assistance
  • Classification and extraction
  • Long-context summarization on sufficiently large memory systems
  • Commercial deployments needing Apache-2.0 licensing

Choose something else when

  • You need vision or multimodal input
  • You only have 8GB RAM and want long-context use
  • You need the smallest possible edge model
  • You need more than the supported language set

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB system memory
Memory or response time may limit use. Try a smaller model first.
16GB system memory
Memory or response time may limit use. Try a smaller model first.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
Memory or response time may limit use. Try a smaller model first.
12GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
8.79B
Input
Text
Published context limit
131,072 tokens
Model format
Safetensors · GGUF · Ollama
Suggested compression
Q4_K_M for most enterprise local deployments; Q5_K_M on 16GB+ VRAM or 24GB+ RAM/unified-memory systems.
Typical Q4 file
5.35GB before runtime and context overhead
Languages
Arabic · Chinese · Czech · Dutch · English · French · German · Italian

First setup

Ollama

ollama run ibm/granite4.1:8b

LM Studio: Use the official `ibm-granite/granite-4.1-8b-GGUF` repository and choose Q4_K_M for broad local compatibility.

llama.cpp

Use a standard text GGUF loader, for example llama-server -hf ibm-granite/granite-4.1-8b-GGUF:Q4_K_M.

Conditions that change the answer

  • The model is marketed as 8B, while Ollama/official GGUF metadata reports about 8.79B to 9B parameters.
  • Although the context window is supported as long-context, using very large context lengths materially increases conversation memory memory.
  • RAM and VRAM estimates are derived from GGUF/Ollama package size plus conservative overhead; conversation memory, context length, batch size, runtime and GPU offload choices can materially change memory needs.

Read the controlling license text →