Who should use this model?
- Local code completion and fill-in-the-middle workflows
Choose something else when
- You need an instruction/chat model; the official model card says the base model is not an instruction model.
What hardware makes sense?
The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.
- CPU only
- Possible but slow
- 8GB system memory
- Use a smaller model for this memory limit.
- 16GB system memory
- Memory or response time may limit use. Try a smaller model first.
- 32GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB GPU memory
- Use a smaller model for this memory limit.
- 12GB GPU memory
- Memory or response time may limit use. Try a smaller model first.
- 24GB GPU memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- Apple unified memory
- 16GB unified memory possible 32GB recommended
Model facts that change the decision
- Parameters
- 15B
- Input
- Text
- Published context limit
- 16,384 tokens
- Model format
- safetensors · transformers · GGUF · Ollama
- Suggested compression
- Q4_K_M
- Typical Q4 file
- 9.86GB before runtime and context overhead
- Languages
- 600+ programming languages
First setup
Ollama
ollama run starcoder2:15bLM Studio: Search for `second-state/StarCoder2-15B-GGUF` and choose Q4_K_M for the first local run.
llama.cpp
Use llama-cli -hf second-state/StarCoder2-15B-GGUF:Q4_K_M -p '<code prefix>'
Conditions that change the answer
- The official StarCoder2 base checkpoints are code-focused but not instruction-tuned; choose an instruction-tuned version if you need chat.
- RAM and VRAM estimates are conservative estimates from GGUF file sizes plus local runtime and conversation memory overhead.
Read the controlling license text →