AnythingLLM is useful when you want to organize documents into a reusable workspace and ask questions supported by their contents. Start with AnythingLLM Desktop for one person on one computer. Use its self-hosted server edition when you need browser access, separate accounts, or shared workspaces.
Start with the two-file exercise below before importing a large library or enabling agents. Check the supporting passage, its qualifications, and questions the documents cannot answer.
Choose where the workspace will live
| Deployment | Suitable starting use | What you maintain |
|---|---|---|
| Desktop | Personal document work on one computer | Application, local data and selected providers |
| Self-hosted Docker | Browser access, multiple users and workspace permissions | Server, persistent storage, access controls and updates |
| AnythingLLM Cloud | A managed remote workspace | Account, provider choices and acceptable remote data handling |
Self-hosting describes who administers the service, not necessarily where the computer is. Docker on a cloud server stores application data remotely. The managed Cloud service reduces server administration but is still remote.
Desktop's network-discovery setting exposes an API rather than a browser interface. It does not turn Desktop into the multi-user edition, and its documentation warns that the service has no user-account authentication. Keep it off for personal use and don't expose it to the public internet.
Understand the work behind a document answer
AnythingLLM first extracts readable text. An embedding model then converts chunks of that text into numerical representations and stores them in a searchable index. When you ask a question, retrieval selects relevant passages, and the answer model writes from the material it receives. This is retrieval-augmented generation, or RAG.
A filename in the library does not establish that the document is indexed in your workspace. A source reference does not establish that its passage supports the answer. A fluent summary does not establish that every page was reviewed.
Keep originals available. A scanned PDF needs suitable optical character recognition, or OCR, to produce text. Check important names, dates, numbers, and table relationships against the image before indexing. If extraction leaves out a condition, the answer model may never receive it.
Configure the separate providers
Install the matching Desktop package through the installation overview. For the first trial, choose the built-in local chat provider, built-in local embedding provider, and LanceDB search storage. These choices avoid setting up separate services, but their initial downloads still require connectivity.
Desktop v1.18.0 introduced Lemond, the embedded Lemonade engine, replacing the previous embedded Ollama engine for new installations. Existing users can be offered an engine migration. Accepting it removes the old engine and its downloaded models, so you'll need to download those models again; the release notes say workspaces, chats, and documents remain. A separately installed Ollama remains a distinct provider option.
The built-in provider accepts compatible GGUF model files and can use supported CPU or GPU paths. Choose a modest model offered for your hardware before importing an arbitrary file. The model's download size isn't its full memory requirement: instructions, conversation, document passages, and working calculations also need room.
The built-in embedding model is sentence-transformers/all-MiniLM-L6-v2, primarily trained on English material. It runs on the CPU, and the documentation advises at least 2 GB of available RAM for that process. This is additional planning for embedding, not a total-system requirement or a guarantee that the answer model fits. For non-English documents, this model may produce lower-quality embeddings, limiting the effectiveness of semantic search and retrieval. Test representative material before relying on it in another language.
Check the workspace's answer-model override as well as the system default. Agents can also have their own model settings. A local system default alone does not establish the destination of every request.
Docker has separate version requirements. Its rolling documentation describes an optional built-in model service for the mutable latest image and releases after v1.18.0. Do not assume a container pinned to v1.18.0 includes it because Desktop changed engines in that release. Use a working external local provider or a server release that explicitly includes the needed feature. Check the edition you actually deploy: read its release notes for engine changes, confirm each required feature in the selected version or Docker image, and check the requirements of its connected providers. Desktop and Docker are alternative deployments; the documentation does not require installing both or matching their version numbers. For a pinned Docker image, use that release's documented capabilities rather than assuming it matches Desktop or the changing latest image. Before upgrading, back up workspaces and test representative files in the new setup.
Build a collection with two checkable files
Create two UTF-8 text files with the following fictional material. A simple text editor is sufficient.
Save this as Harbour_Library_Hours.txt:
Harbour Library visitor hours
Fictional example. Effective date: 1 October 2026.
The library opens at 09:15 and closes at 17:30 Monday through Friday.
On Saturdays it opens at 10:00 and closes at 13:00.
The library is closed on Sundays.
These hours do not describe the community room's booking rules.
Save this as Harbour_Community_Room.txt:
Harbour Library community room rules
Fictional example. Effective date: 1 October 2026.
The community room holds at most 12 visitors.
A booking request must be received at least two working days before the event.
The room may be booked only during the library's visitor hours.
These rules do not state a hire price or a cancellation fee.
-
Create a workspace named Harbour practice.
-
Open its settings and choose Query mode under Chat Settings. Newer workspaces may default to Agent mode, which uses enabled tools. Query mode is intended for answers based on uploaded documents; still check its answers.
-
Use the workspace document manager to upload both files. Select them for the workspace and complete the save or embedding action. Wait for processing to finish.
-
Open a new conversation in that workspace and ask the questions below separately.
A direct chat attachment is a different route. It ordinarily supplies its text to that conversation; embedding makes a document searchable across workspace threads. Do not assume a file visible in My Documents is already available to the current workspace.
| Question to ask | What the fictional sources support |
|---|---|
| When does the library open on Saturday? Quote the sentence and name the file. | 10:00, from Harbour_Library_Hours.txt. |
| Can a group of 14 visitors book the room? Give the rule and filename. | The room's limit is 12 visitors. |
| Could 10 visitors use the room at 14:00 on Saturday? Explain using both files. | Ten is within capacity, but Saturday closing is 13:00 and bookings must be during visitor hours. |
| What is the room's hire price? | The files do not state it. |
These are answer checks derived from invented inputs, not model results. Open each supporting passage and compare it with the original. The third question needs both documents. If only one were retrieved, ask the narrower questions first; do not accept a confident combined answer without support.
Start another thread and repeat a question. Then reopen AnythingLLM and try again. This helps establish that the reusable collection, rather than an earlier chat attachment or answer, supplies the information.
Improve retrieval before increasing model size
If the right passage is missing, confirm the workspace, document membership, completed embedding, and extracted text. Ask using distinctive words from the source. A larger answer model is not the first repair for missing source material.
In workspace Vector Database Settings, inspect Document similarity threshold and Max Context Snippets. A restrictive threshold can exclude a useful match. On the harmless example, compare with No restriction and inspect whether irrelevant passages appear. More snippets may recover a condition but use more context and add noise.
With LanceDB, Search Preference includes an Accuracy Optimized option that reranks candidate passages. Reranking adds work. Compare the same questions and source support before keeping the setting; its name doesn't guarantee every answer becomes accurate.
When a passage cuts a rule away from its exception, inspect Settings → AI Providers → Text Splitter & Chunking. The documented defaults are 1,000 characters per chunk and 20 characters of overlap. These are characters, not tokens. Larger chunks keep more surrounding explanation but bring more unrelated text. Smaller chunks can separate facts that need to be interpreted together. Changes affect newly embedded documents, so re-embed the test files before comparing results.
Changing the embedding model is more consequential. The setting is system-wide, and you need to re-embed existing documents with the new model. Keep originals and plan the whole collection change rather than switching providers halfway through an important review.
For a short critical document, full-text attachment or document pinning may be useful when all the text fits. Pinning inserts an already embedded document's full text into context. Neither method guarantees correct interpretation. For an exhaustive review of a long document, work through sections explicitly and maintain your own coverage checklist; retrieval selects passages rather than certifying complete review.
Keep revisions and deletion distinct
Editing an original file does not by itself establish that the workspace has its new contents. Try a controlled change: revise Saturday closing to 15:00, save a clearly named replacement, remove the obsolete workspace version, add and process the replacement, then ask the 14:00 question in a fresh thread. Check the new passage. Leaving both versions indexed can produce contradictory answers.
Automatic Document Sync is documented as a feature preview. Its supported sources differ between Desktop and Docker. A browser upload does not give a remote Docker server continuing access to a file on your computer. Moving or deleting a watched local original can stop monitoring while previously indexed content remains. Confirm the actual refreshed text instead of relying on a watch indicator or a presumed sync interval.
Removing a document from one workspace deletes that workspace's vectors while leaving the library copy in My Documents. Deleting it from My Documents removes the parsed source and cached embeddings and removes it from every workspace. Neither action establishes deletion of the original, old answers quoting it, exports, backups or copies held by external providers.
Protect storage and sharing
On Desktop, use the version menu's Open storage command to locate the active data folder. It contains more than models: application records, parsed documents, search data, direct uploads, plugins and logs may matter. A chat export is not a complete collection backup. Preserve originals separately, protect backup copies, and test restoration in a separate installation before relying on a recovery method. Remote databases and services require their own arrangements.
Review extraction, embeddings, search storage, answer generation, tools and sync as separate destinations. A local embedding provider and answer model keep those two processing steps local; they do not establish where extraction, vector storage, tools or synchronization send data. Check those destinations and the location of the application server too. A cloud embedding provider receives document text even when the answer model is local, and a remote answer model receives your question and the retrieved passages. Uploading through a browser interface to a remote server transfers the files, and some plugins and skills send data outside your environment. Telemetry can be disabled in settings; doing so does not disable downloads, cloud providers, or tool traffic.
For a shared Docker installation, use appropriate user roles and workspace membership. A shared single-user password gives broad access; multi-user mode provides separate roles, and its documentation says the mode switch cannot be reversed. Test with an ordinary account and an account that should lack access.
Agents can use enabled skills and tools, some of which may already be available by default. Review Agent Skills and the chat's Tools controls before enabling them. Web, mail, API, and file tools can read, send, or change information. Treat instructions inside a document as source content, not authorization to take those actions.
If publishing a website chat widget, use a separate collection intended for public use. Domain restrictions and usage limits do not make confidential workspace documents suitable for publication. Protect API keys; they enable programmatic access.
Recover from the failing stage
| Problem | What to try first |
|---|---|
| Ordinary chat works but a document fails to process | Check the embedding provider's initial download, available memory and error. Retry the small text file. |
| A Docker app cannot reach a host model | Check the host address. Inside a container, localhost refers to the container. Docker Desktop commonly uses host.docker.internal; native Linux needs its own configuration. |
| A model is slow or runs out of memory | Reduce unnecessary context and competing models, or try smaller weights. Keep information essential to the task. |
| A convincing answer cites the wrong passage | Stop relying on it. Separate retrieval from interpretation and inspect the original yourself. |
| An updated file still produces old facts | Check the indexed version and use a new thread; replace and reprocess the intended source. |
After an application or engine update, repeat ordinary chat, the two-file questions and the reopened-workspace check before returning to important material. Keep the configuration that passed those checks identifiable.
AnythingLLM's application source uses the MIT License. The embedding model's model card identifies Apache 2.0. Answer models, connected services, and optional paid offerings have separate terms. An application's license does not grant rights to everything it can download or connect to.
Related pages
Sources
-
Desktop v1.18.0 changes, built-in model provider and Docker built-in service requirements.
-
Document handling and retrieval settings, chat modes, chunking, embedding configuration and default embedder.
-
Document Sync preview, storage, data handling and Desktop privacy policy.
-
Server deployment, managed Cloud, access controls, agent setup and chat widgets.
For the broader PDF workflow, see Chat with PDFs locally. For deployment choices, see Open WebUI vs AnythingLLM.