Who should use this model?
- Fine-tuning higher-quality compact classifiers, routers, token taggers, retrieval systems, and similarity models.
Choose something else when
- You want a ready-made chat assistant or the smallest encoder in this family.
What hardware makes sense?
The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. Document-search memory also depends on input length, batch size, and the document index; a chat-memory estimate does not confirm that workload.
- CPU only
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 16GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 32GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB GPU memory
- Not required
- 12GB GPU memory
- Not required
- 24GB GPU memory
- Not required
- Apple unified memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
Model facts that change the decision
- Parameters
- 0.3545B
- Input
- Text
- Published context limit
- 8,192 tokens
- Model format
- safetensors · Transformers
- Languages
- English · German · Spanish · French · Italian · Dutch · Polish · Portuguese
Adapting an encoder
This encoder is intended for tasks such as classification or retrieval after suitable adaptation. Follow its model card rather than assuming that a chat app or an embedding app can use it unchanged.
First setup
llama.cpp
The official card uses custom Transformers code; do not assume llama.cpp support.
Conditions that change the answer
- This is a masked-language-model encoder, not a generative chat model.
- The official card requires trust_remote_code=True; review remote code before use.
- The custom license includes a commercial-use revenue threshold.
Read the controlling license text →