Who should use this model?
- Structured extraction, tool routing, and lightweight on-device agent components.
Choose something else when
- You need advanced reasoning, code generation, or creative writing.
What hardware makes sense?
The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.
- CPU only
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 16GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 32GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB GPU memory
- Not required
- 12GB GPU memory
- Not required
- 24GB GPU memory
- Not required
- Apple unified memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
Model facts that change the decision
- Parameters
- 0.23B
- Input
- Text
- Published context limit
- 32,768 tokens
- Model format
- safetensors · GGUF · ONNX · MLX · Transformers · vLLM · SGLang
- Suggested compression
- Use an official GGUF build for llama.cpp-compatible local inference.
- Languages
- English · Arabic · Chinese · French · German · Italian · Japanese · Korean
First setup
LM Studio: Import a compatible official artifact and verify the selected runtime.
llama.cpp
Use the official GGUF repository with a current llama.cpp build.
Conditions that change the answer
- The custom license limits commercial use for entities at or above $10 million in annual revenue.
- The official card does not recommend this model for reasoning-heavy math, code generation, or creative writing.
Read the controlling license text →