Skip to main content
Local LLM Guide

Model

LLaVA 1.6 Vicuna 13B

Start with a supported compressed build and short context. 32GB RAM estimate. Best use: Higher-quality local image chat than the 7B LLaVA 1.6 variant

13BVision language

Best use

Higher-quality local image chat than the 7B LLaVA 1.6 variant · Commodity workstation multimodal demos

Memory planning

32GB RAM estimate

Expected pace

Depends on your hardware, app, and conversation length

License

Llama 2 Community License; read the model’s terms before use

Who should use this model?

  • Higher-quality local image chat than the 7B LLaVA 1.6 variant
  • Commodity workstation multimodal demos
  • Research or hobbyist visual question-answering
  • Testing larger LLaVA-compatible local runtimes

Choose something else when

  • You only have 8GB RAM or 8GB VRAM
  • You need production OCR accuracy
  • You need permissive Apache/MIT-style commercial licensing
  • You need tool calling or long text context

What hardware makes sense?

The download is only part of the memory needed. Leave room for the app, the input it processes, and other open programs. For chat models, context includes the instructions, conversation, and answer. Longer context needs more memory.

CPU only
Memory or response time may limit use. Try a smaller model first.
8GB system memory
Use a smaller model for this memory limit.
16GB system memory
Memory or response time may limit use. Try a smaller model first.
32GB system memory
A possible starting point with a compressed model and short conversation; check the exact file size.
8GB GPU memory
Use a smaller model for this memory limit.
12GB GPU memory
Memory or response time may limit use. Try a smaller model first.
24GB GPU memory
A possible starting point with a compressed model and short conversation; check the exact file size.
Apple unified memory
A possible starting point with a compressed model and short conversation; check the exact file size.

Model facts that change the decision

Parameters
13B
Input
Text · Image
Published context limit
4,096 tokens
Model format
Hugging Face Transformers · GGUF · Ollama
Suggested compression
Q4_K_M for local image-chat on 16GB+ systems; Q5_K_M if 32GB RAM or ample unified memory is available.
Typical Q4 file
7.87GB before runtime and context overhead
Languages
English

Using image input

This model needs an app or library that supports its image-processing components. Follow the exact model’s instructions for preparing image inputs. Some GGUF downloads also need a matching projector file; loading the text model alone does not establish image support.

First setup

Ollama

ollama run llava:13b-v1.6-vicuna-q4_K_M

LM Studio: Use a LLaVA-capable GGUF workflow and ensure the matching multimodal projector is loaded if required.

llama.cpp

Use a current llama.cpp LLaVA/multimodal binary and pass the model plus matching --mmproj projector file.

Conditions that change the answer

  • Community GGUF conversion is not an official LLaVA artifact.
  • The 13B model is materially less suitable for 8GB systems than the 7B variant.
  • Local vision inference may require a separate projector/mmproj file depending on runtime.
  • RAM and VRAM estimates are derived from GGUF/Ollama package size plus conservative overhead; conversation memory, image resolution, batch size, GPU offload and runtime support can materially change memory needs.

Read the controlling license text →