Who should use this model?
- Multilingual document search and semantic search
Choose something else when
- You need a chat or code-generation model.
What hardware makes sense?
The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. Document-search memory also depends on input length, batch size, and the document index; a chat-memory estimate does not confirm that workload.
- CPU only
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 16GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 32GB system memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 8GB GPU memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 12GB GPU memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- 24GB GPU memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
- Apple unified memory
- A possible starting point with a compressed model and short conversation; check the exact file size.
Model facts that change the decision
- Parameters
- 0.567B
- Input
- Text
- Published context limit
- 8,192 tokens
- Model format
- safetensors · sentence-transformers · PyTorch · ONNX · GGUF · Ollama
- Suggested compression
- Q8_0 for quality; Q4_K_M for minimal local footprint
- Typical Q4 file
- 0.438GB before runtime and context overhead
- Languages
- 100+ working languages
Using this model for document search
Select this model for the document-search step in an app that supports it. Keep the model that generates answers separate. Check document chunk length and any required query or document prefixes in the model’s instructions.
Changing the embedding model changes the document vectors. Rebuild the document index rather than mixing vectors from different models.
First setup
Download with Ollama
ollama pull bge-m3LM Studio: Use an embedding workflow, not chat
llama.cpp
Use an embedding-capable llama.cpp build/runtime and a vetted GGUF such as lm-kit/bge-m3-gguf
Conditions that change the answer
- BGE-M3 is an embedding/retrieval model, not a chat or code-generation model.
Read the controlling license text →