Who should use this model?
- Multilingual retrieval where higher accuracy matters more than index size.
Choose something else when
- You need the smallest possible index or a simple single-vector embedding pipeline.
What hardware makes sense?
The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. Document-search memory also depends on input length, batch size, and the document index; a chat-memory estimate does not confirm that workload.
- CPU only
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 16GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 32GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB GPU memory
- Not required
- 12GB GPU memory
- Not required
- 24GB GPU memory
- Not required
- Apple unified memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
Model facts that change the decision
- Parameters
- 0.353B
- Input
- Text
- Published context limit
- 512 tokens
- Model format
- safetensors · GGUF · llama.cpp
- Suggested compression
- Use an official artifact supported by the chosen retrieval runtime.
- Languages
- Arabic · German · English · Spanish · French · Italian · Japanese · Korean
Using this model for document search
Select this model for the document-search step in an app that supports it. Keep the model that generates answers separate. Check document chunk length and any required query or document prefixes in the model’s instructions.
Changing the embedding model changes the document vectors. Rebuild the document index rather than mixing vectors from different models.
First setup
llama.cpp
Use the official GGUF repository with current reranking support.
Conditions that change the answer
- Per-token vectors create a larger index than the companion embedding model.
- The 512-token document limit requires chunking longer material.
- The custom license includes a commercial-use revenue threshold.
Read the controlling license text →