Who should use this model?
- Fine-tuning compact classifiers, routers, token taggers, retrieval systems, and similarity models.
Choose something else when
- You want a ready-made chat assistant or drop-in retriever without adaptation.
What hardware makes sense?
The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. Document-search memory also depends on input length, batch size, and the document index; a chat-memory estimate does not confirm that workload.
- CPU only
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 16GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 32GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB GPU memory
- Not required
- 12GB GPU memory
- Not required
- 24GB GPU memory
- Not required
- Apple unified memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
Model facts that change the decision
- Parameters
- 0.2297B
- Input
- Text
- Published context limit
- 8,192 tokens
- Model format
- safetensors · Transformers
- Languages
- English · German · Spanish · French · Italian · Dutch · Polish · Portuguese
Adapting an encoder
This encoder is intended for tasks such as classification or retrieval after suitable adaptation. Follow its model card rather than assuming that a chat app or an embedding app can use it unchanged.
First setup
llama.cpp
The official card uses custom Transformers code; do not assume llama.cpp support.
Conditions that change the answer
- This is a masked-language-model encoder, not a generative chat model.
- The official card requires trust_remote_code=True; review remote code before use.
- The custom license includes a commercial-use revenue threshold.
Read the controlling license text →