Use this glossary when a guide mentions model size, quantization, context length, RAM, VRAM, embeddings, or RAG. The safest beginner rule is that every term depends on the exact model file, runtime, hardware, and app settings.
Core terms
| Term | Plain-English meaning | Why it matters |
|---|---|---|
| Local AI | An AI workflow where some or all processing happens on hardware you control. | A local app can still call cloud services, so check the full workflow. |
| Local LLM | A language model file running through a local runtime or app. | The model may stay local, but tools, embeddings, updates, or web features can still use the network. |
| Model runner (runtime) | Software that loads and runs model files. | Ollama and llama.cpp-style backends are runtime examples; interface apps may use a runtime underneath. |
| Desktop app | A graphical app for local AI workflows. | A desktop app can be easier than a terminal, but privacy still depends on providers and settings. |
| Model file | The downloaded weights used to generate responses. | File size, format, license, and quantization affect what can run on your machine. |
| Parameters | A rough size measure for a model, often written as 3B, 7B, 14B, or 70B. | Larger models usually need more memory and storage. Bigger is not automatically better for a beginner setup. |
| Model compression (quantization) | A way to store model weights with fewer bits, often shown as Q4, Q5, or Q8. | Lower-bit quantization can reduce memory use, often with quality tradeoffs. |
| Context length | How much text the model can consider at once. | Long context can use much more memory, especially during document chat. |
| RAM | General system memory. | CPU-only and many app workflows use RAM, and the operating system needs some too. |
| VRAM | Dedicated GPU memory. | GPU-accelerated local AI is often limited by dedicated VRAM, not just total system RAM. |
| Unified memory | A shared memory pool accessible to the CPU and GPU, used by Apple Silicon and some other integrated designs. | The OS, apps, and model share available capacity; it is not a separate dedicated VRAM pool. |
Document-chat terms
| Term | Plain-English meaning | Why it matters |
|---|---|---|
| Embeddings | Numeric representations of text used for search and retrieval. | If embeddings are created by a cloud provider, document text may leave your machine. |
| RAG | Retrieval augmented generation: search documents first, then send relevant text to the model. | RAG can help with PDFs, but it can still retrieve the wrong chunks or miss important context. |
| Chunking | Splitting documents into smaller pieces for retrieval. | Bad chunking can make answers incomplete or misleading. |
| Vector database | Storage used to search embeddings. | It may be local or remote depending on the app and configuration. |
| OCR | Optical character recognition for scanned or image-heavy PDFs. | A model cannot answer from text it never receives; scanned PDFs may need OCR first. |
| Citation | A pointer to a retrieved document passage or source. | Treat citations as pointers, not proof. Always check the original document. |
Privacy and setup terms
| Term | Plain-English meaning | Why it matters |
|---|---|---|
| Local provider | A model or embedding provider running on your machine. | This is usually required for a local-only workflow. |
| Hosted provider | A cloud API or remote model service. | Prompts, files, or document chunks may leave your machine. |
| Local server | A service running on your computer, often at localhost. | If exposed to the network, the risk profile changes. |
| Telemetry | Usage or diagnostic data sent by an app. | Review privacy settings before using sensitive files. |
| Benchmark | A measured performance result from a specific machine, model, and setup. | A memory estimate is not a measured speed result. |
| Compatibility | Whether a setup works with a specific model, runtime, operating system, and hardware. | Compatibility cannot be guaranteed from RAM or VRAM alone. |
How to use the terms safely
- Start with the hardware terms: RAM, VRAM, unified memory, model size, quantization, and context length.
- Then check the workflow terms: runtime, app, local provider, hosted provider, embeddings, and RAG.
- Treat every estimate as setup guidance, not proof.
- Use a harmless first test file before sensitive document workflows.
- Check official documentation when a setting name, model catalog, or install command differs from a guide.