Quick diagnosis
A local PDF workflow has four separate jobs: extract the text, split and index it, retrieve the right passages, and let the model answer from those passages. When PDF chat fails, identify which job failed before changing the model.
A larger chat model does not repair missing text, bad reading order, weak retrieval, or an embedding service pointed at the wrong provider.
First three checks
- Inspect extracted text from one short, text-based PDF and confirm a distinctive sentence survived.
- Confirm indexing completed, then search for that sentence and inspect the retrieved passage.
- Ask a question answered by that passage; if the answer still fails, check the prompt and available context.
The four-stage diagnostic
| Stage | What success looks like | Common failure |
|---|---|---|
| Extraction | The text is readable and complete in the correct order | Scanned pages, columns, tables, or fonts produce missing or scrambled text |
| Indexing | The document is split and stored without silent errors | Wrong parser, empty chunks, failed embeddings, or stale index |
| Retrieval | A direct question returns the passage that contains the answer | Chunks are too large or small, embeddings are weak, or search settings miss the passage |
| Answer | The model uses the retrieved passage and stays within it | Too little context, poor prompting, or unsupported conclusions |
Test each stage independently. Do not jump from “the answer is wrong” directly to “the model is bad.”
Where to look differs by app:
| Stage | LM Studio | Open WebUI | AnythingLLM |
|---|---|---|---|
| Accepted file types | .pdf, .docx, .txt | Extraction engines include Tika, Docling and Mistral OCR; its retrieval documentation does not name the default engine | Check AnythingLLM's own documentation for accepted file types |
| Full text or retrieval | Automatic: LM Studio adds the whole file when it fits in the context, and uses retrieval when it does not | Settings → Admin → Documents sets the embedding model and chunking. For one file or knowledge base, the “Using Entire Document” toggle skips retrieval | In workspace Vector Database Settings, Document similarity threshold and Max Context Snippets. Settings → AI Providers → Text Splitter & Chunking sets chunk size (default 1,000 characters) and overlap (default 20 characters) for documents embedded afterwards |
| See what was retrieved | LM Studio's documentation does not say it shows retrieved passages. Test by asking for a quotation from a sentence you know is in the file | Open the source passages shown with the answer | Open the cited passage under the answer |
| Rebuild after a change | LM Studio's documentation does not describe this | Settings → Admin → Documents → Reindex after changing the embedding model; re-upload standalone chat files | Remove the document and embed it again; new chunking settings do not change documents already embedded |
Stage 1: inspect the extracted text
Open or preview the text that the app actually extracted (the Open WebUI and AnythingLLM pages say where). Check:
- whether every relevant page produced text;
- whether headers, footers, and page numbers overwhelm the body;
- whether two-column layouts are interleaved;
- whether tables preserve row and column relationships;
- whether ligatures, symbols, or non-English text are corrupted;
- whether a scanned PDF needs OCR;
- whether attachments or embedded images contain important content.
Ask the system to retrieve a distinctive sentence that appears verbatim in the source. If the sentence is absent from the extracted text, retrieval tuning cannot fix the problem.
Stage 2: confirm indexing and embeddings
A document can appear in the interface while its index is incomplete or stale. Confirm that:
- the document finished processing;
- chunks contain real text;
- the embedding model ran locally when that is your intended boundary;
- the index belongs to the current document version;
- re-importing did not create conflicting duplicates;
- the vector store or search index is available to the current workspace.
Use one small document first. A large library hides basic ingestion failures behind volume.
Stage 3: test retrieval before generation
Ask a question whose answer appears in one clear passage. Inspect the passages returned by search or retrieval before the final model writes an answer.
If the right passage is not returned, work on retrieval:
- try a more literal query;
- adjust chunk size and overlap;
- preserve useful page, heading, or section metadata;
- remove repeated headers and footers;
- compare semantic search with keyword or hybrid search;
- use a reranker only after basic retrieval works;
- separate documents that use the same terms for different subjects.
If the right passage is returned near the top, retrieval is probably not the primary failure.
Stage 4: constrain the answer
When retrieval works but the answer is poor:
- instruct the model to answer from the retrieved passages;
- require it to say when the passages do not contain the answer;
- include enough surrounding text to preserve meaning;
- avoid stuffing many weak passages into the prompt;
- keep the question specific;
- use a model that can reliably follow grounding instructions;
- display the supporting passage or page reference beside important answers.
A fluent answer without the right source passage is not a successful document workflow.
Diagnose by symptom
The PDF uploads but produces no useful text
The source may be scanned, image-based, encrypted, malformed, or difficult for the selected parser. Run OCR where appropriate and inspect the result before indexing it.
Answers mention the document but miss obvious facts
Test retrieval with an exact phrase. If the passage is missing, fix extraction or retrieval. If the passage is present, tighten the answer prompt and reduce distracting context.
Tables produce nonsense
Plain text extraction often loses table structure. Use a parser that preserves layout or extract important tables separately into a structured format. Do not expect a language model to reconstruct missing row relationships reliably.
The first document works but a library does not
The index may be mixing documents, retrieving repeated boilerplate, or exceeding practical memory and context. Add metadata filters, inspect top results, and scale the library only after a small multi-document test works.
The workflow sends data to a remote provider
Check both the chat model and the embedding model. Also inspect OCR, parser, reranker, web search, sync storage, and any plugin or connector involved in document ingestion.
A minimal working test
- Choose a short, text-based PDF with one distinctive sentence.
- Import it and inspect the extracted text.
- Confirm that the distinctive sentence appears intact.
- Search for that sentence or a close paraphrase.
- Inspect the passages returned before generation.
- Ask one question answered directly by the passage.
- Confirm the final answer points back to the correct passage or page.
- Only then add a scanned PDF, table-heavy document, or larger library.
What success looks like
The system extracts readable text, retrieves the relevant passage for a targeted question, produces an answer that stays within the retrieved material, and lets the user inspect the support. The same result should survive a normal restart and a clean re-import of the test document.
When to stop and change tools
Switch parsers or workflows when:
- scanned pages require OCR the current tool does not provide;
- tables or columns lose essential structure;
- the interface does not let you inspect retrieved passages;
- the workflow cannot keep embeddings and documents within the intended data boundary;
- a large library is being used to conceal a failure that also occurs on one small test document.
Next step
Use the local PDF setup guide after the four-stage test works. For sensitive material, also map the complete local AI privacy workflow, including embeddings, indexes, storage, and sync.