Skip to main content
Local LLM Guide

Model

Phi-3 Mini 3.8B Instruct

Start with a supported compressed build and short context. 8GB RAM estimate. Best use: small local chat

3.8BText chat

Best use

small local chat · math and logic prompts

Memory planning

8GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

MIT; read the model’s terms before use

Who should use this model?

  • small local chat
  • math and logic prompts
  • latency-sensitive local assistants
  • laptop inference

Choose something else when

  • a context window above 4K is required for the selected checkpoint
  • multimodal input is required
  • broad multilingual support is required

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
16GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
12GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
3.8B
Input
Text
Published context limit
4,096 tokens
Model format
safetensors · GGUF · Ollama · Transformers · ONNX
Suggested compression
Q4_K_M for default local use; Q5_K_M if it still fits comfortably in RAM/VRAM.
Typical Q4 file
2.393GB before runtime and context overhead
Languages
English

First setup

Ollama

ollama run phi3:mini

LM Studio: Search for microsoft/Phi-3-mini-4k-instruct-gguf or tensorblock/Phi-3-mini-4k-instruct-GGUF and select Q4_K_M.

llama.cpp

Use the Microsoft official GGUF when possible; use the TensorBlock community GGUF only when a specific K-quant file is needed.

Conditions that change the answer

  • This record uses the official Microsoft 4K instruct checkpoint because it has an official Microsoft GGUF repository; Phi-3 Mini also has a 128K variant.
  • The current Microsoft Hugging Face card describes a June 2024 updated checkpoint with 4.9T training tokens, while earlier official launch/GGUF material described the original release with 3.3T tokens.
  • Ollama's current Phi-3 page mixes Mini tags and 128K tags; verify the selected tag before relying on context length.
  • Allow memory for the model file, the app, and your conversation.

Read the controlling license text →