Skip to main content
Local LLM Guide

Model

Mistral Small 3.2 24B Instruct 2506

Start with a supported compressed build and short context. 32GB RAM estimate. Best use: high-quality local 24B-class chat

24BText chat

Best use

high-quality local 24B-class chat · vision-capable local assistant workflows

Memory planning

32GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

Apache-2.0; read the model’s terms before use

Who should use this model?

  • high-quality local 24B-class chat
  • vision-capable local assistant workflows
  • function-calling and instruction-following use
  • single-4090 or 32 GB unified-memory class deployments

Choose something else when

  • you need a text-only model card with no multimodal runtime caveats
  • you have 16 GB RAM or less
  • your runtime cannot apply the required chat template for tool calling

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
Memory or response time may limit use. Try a smaller model first.
8GB system memory
Use a smaller model for this memory limit.
16GB system memory
Memory or response time may limit use. Try a smaller model first.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
Use a smaller model for this memory limit.
12GB GPU memory
Some model work must use system memory, which can slow responses.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
Comfortable on 32GB or more unified memory

Model facts that change the decision

Parameters
24B
Input
Text · Image
Published context limit
128,000 tokens
Model format
safetensors · GGUF · Ollama
Suggested compression
Q4_K_M
Typical Q4 file
14.33GB before runtime and context overhead
Languages
English · French · German · Spanish · Italian · Portuguese · Japanese · Korean

Using image input

This model needs an app or library that supports its image-processing components. Follow the exact model’s instructions for preparing image inputs. Some GGUF downloads also need a matching projector file; loading the text model alone does not establish image support.

First setup

Ollama

ollama run mistral-small3.2:24b-instruct-2506-q4_K_M

LM Studio: Use bartowski or lmstudio-community GGUF builds for Mistral-Small-3.2-24B-Instruct-2506; select Q4_K_M on 24 GB VRAM or 32 GB unified-memory systems.

llama.cpp

Use llama.cpp with the Q4_K_M GGUF and the Mistral Small 3.2 Jinja chat template when tool calling is required.

Conditions that change the answer

  • Memory needs include the model file, the app, and conversation memory. Longer conversations and other open apps need more space.
  • This target uses Mistral Small 3.2 rather than 3.1 because the 3.2 2506 artifact has a clear official model card, current Ollama tag, and multiple GGUF listings.
  • Vision support requires a runtime that supports the model's multimodal inputs; text-only runtimes may expose only text chat.
  • Some GGUF notes indicate tool calling depends on using the correct Jinja chat template.
  • Official Mistral documentation lists this 2506 model with a deprecation date of 2026-04-30 and a replacement path to Mistral Small 4 for hosted/API model selection; local GGUF artifacts remain available.

Read the controlling license text →